-- Logs begin at Thu 2022-09-15 10:16:13 UTC, end at Thu 2022-09-15 11:41:13 UTC. -- Sep 15 10:41:33.860691 dv-cinder-824149-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 15 10:41:33.868592 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 15 10:41:33.870603 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 15 10:41:33 2022] *** Sep 15 10:41:33.870603 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 15 10:41:33.870603 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 15 10:41:33.870603 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: nodename: dv-cinder-824149-8-iscsi Sep 15 10:41:33.870603 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: machine: x86_64 Sep 15 10:41:33.870603 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: clock source: unix Sep 15 10:41:33.870603 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: pcre jit disabled Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: detected number of CPU cores: 4 Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: current working directory: / Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: detected binary path: /usr/bin/uwsgi-core Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: your processes number limit is 47680 Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: your memory page size is 4096 bytes Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: detected max file descriptor number: 2048 Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: lock engine: pthread robust mutexes Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: thunder lock: enabled Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 15 10:41:33.871054 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 15 10:41:33.910832 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: Python main interpreter initialized at 0x555a92eed9c0 Sep 15 10:41:33.910832 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: python threads support enabled Sep 15 10:41:33.910832 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: your server socket listen backlog is limited to 100 connections Sep 15 10:41:33.910832 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: your mercy for graceful operations on workers is 90 seconds Sep 15 10:41:33.911793 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: mapped 403077 bytes (393 KB) for 2 cores Sep 15 10:41:33.911793 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: *** Operational MODE: preforking *** Sep 15 10:41:33.912082 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: *** uWSGI is running in multiple interpreter mode *** Sep 15 10:41:33.912082 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: spawned uWSGI master process (pid: 81376) Sep 15 10:41:33.912286 dv-cinder-824149-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 15 10:41:33.916827 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: spawned uWSGI worker 1 (pid: 81387, cores: 1) Sep 15 10:41:33.916827 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: spawned uWSGI worker 2 (pid: 81388, cores: 1) Sep 15 10:41:33.916827 dv-cinder-824149-8-iscsi devstack@keystone.service[81376]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 15 10:41:38.216456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 10:41:38.216456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: from cryptography.utils import int_from_bytes Sep 15 10:41:38.217282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 10:41:38.217282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: from cryptography.utils import int_from_bytes Sep 15 10:41:38.218654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 10:41:38.218654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: from cryptography.utils import int_from_bytes Sep 15 10:41:38.219119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 10:41:38.219119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: from cryptography.utils import int_from_bytes Sep 15 10:41:38.266775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 10:41:38.266968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 10:41:38.267159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] command line args: [] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 10:41:38.267428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 10:41:38.267606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ================================================================================ {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 10:41:38.267860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] admin_token = **** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.268107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] config_dir = [] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.268295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.268388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 10:41:38.268465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] config_source = [] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.268543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 10:41:38.268642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] debug = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.268724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] command line args: [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 10:41:38.268861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.269105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 10:41:38.269170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ================================================================================ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 10:41:38.269240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.269385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] admin_token = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.269478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.269624 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] config_dir = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.269710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.269814 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.269892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.269995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] config_source = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.270101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] list_limit = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.270199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] debug = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.270309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] log_config_append = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.270407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.270528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.270649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.270754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] log_dir = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.270754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] log_file = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.270946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.271049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.271243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.271878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_config_append = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_dir = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.272941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_file = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.273102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.273262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.273366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.273440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.273562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.273653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.273797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.273944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.274029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.274156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.274235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.274361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.274438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.274554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.274660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.274748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.274899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.275136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.275234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.275371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.275457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] publish_errors = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.275606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.275687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.275843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.275964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.276040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.276154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.276241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.276350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.276474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.276708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] standard_threads = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.276859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.276952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] publish_errors = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] use_journal = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] use_json = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] use_stderr = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.277938 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.278037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] use_syslog = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.278117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.278270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.278367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] standard_threads = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.278524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.278608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.278766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.278846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.278964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.279085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.279183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_journal = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.279261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.279340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_json = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.279466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_stderr = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.279550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.279623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_syslog = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.279706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.279859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 10:41:38.279938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.280115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.280193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.280357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.280453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.280615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.280694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.280882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.280962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.281119 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.281209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.281366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.281445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.281602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.281680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.281845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.281931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.282087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.282165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.282324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.282408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.282569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.282657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.282805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.282889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.283049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.283134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.283280 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.283367 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.283527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.283613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.283774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.283853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.284011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.284169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.284244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.284314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.284476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.284554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.284707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.284785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.284961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.285040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.285196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.285281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.285445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.285530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.285691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.285770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.285921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.285999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.286177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.286262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.286420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.286505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.286653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.286733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.286887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.286971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.287133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.287217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.287361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.287445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.287597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.287676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.287824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.287901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.288052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.288134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.288284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.288363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.288515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.288599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.288748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.288847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.289998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.290151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.290230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.290382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.290461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.290614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.290697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.290847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.290924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.291080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.291157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.291313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.291390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.291540 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.291641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.291774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.291877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.292007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.292108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.292234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.292342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.292474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.292593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.292705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.292851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.292964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.293093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.293228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.293316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.293442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.293523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.293754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.293874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.293960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.294043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.294120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.294236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.294312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.294438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.294553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.294641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.294769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.294850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.295032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.295155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.295272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.295403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.295546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.295623 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.295737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.295812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.295949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.296043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.296195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.296308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.296433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.296513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.296671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.296761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.296936 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.297016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.297173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.297253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.297408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.297496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.297645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.297724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.297877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.297955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.298113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.298201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.298348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.298426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.298579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.298663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.298816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.298906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.299062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.299140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.299299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.299376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.299536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.299622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.299770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.299847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.300007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.300091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.300240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.300330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.300472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.300559 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.300707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.300787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.300966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.301130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.301207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.301284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.301414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.301502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.301670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.301763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.301940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.302102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.302183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.302399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.302500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.302642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.302795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.302873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.302985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.303098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.303243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.303322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.303505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.303586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.303746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.303831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.303978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] auth.external = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.304058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.304211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.304297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.304460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.304540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.304701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.304790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.304974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] auth.password = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.305053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.305206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] auth.token = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.305289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.305445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.305525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.305680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.external = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.305760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.305914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.305995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.306143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.306221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.306389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.306471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.306635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.306716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.password = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.306867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.306946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.token = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.307103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.307181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] credential.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.307338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.307420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.307582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.307661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.307813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.307891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.308061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.308141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.308305 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.308384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.308535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.308613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.308772 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.308938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.309044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.309123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.309278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.309357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.309527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.309607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.309770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.309850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.310015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.310094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.310251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.310332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.310497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.310576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.310749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.310829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.310982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] federation.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.311061 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.311220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.311306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.311465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.311546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.311712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.311793 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.311942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.312022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.312190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.312272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.312428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.312508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.312680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.312765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.312939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.313030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.313187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.313266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.313422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.313500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.313654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.313740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.313894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.313973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.314131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.314212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.314374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.314461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.314610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.314688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.314845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.314923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.315089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.315238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.315321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.315394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.315542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.315632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.315789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.315877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.316034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.316113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.316266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.316344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.316498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.316585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.316731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.316832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.316998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.317077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.317239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.317916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.318000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.318174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.318257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.318438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.318518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.318688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.318776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.318920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.318997 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.319167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.319247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.319416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.319497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.319650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.319730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.319889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.319969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.320122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.320203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.320359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.320438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.320600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.320679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.320864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.320950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.321097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.321176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.321335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.321426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.321568 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.321648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.321805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.321886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.322039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.322118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.322277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.322358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.322520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.322599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.322751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.322838 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.322987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.323067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.323217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.323297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.323455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.323544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.323691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.323772 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.323930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.324017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.324159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.324246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.324394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.324474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.324623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.324702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.324884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.325014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.325124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.325254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.325363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.325489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.325600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.325758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.325878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.325989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.326070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.326227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.326306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.326465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.326547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.326702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.326782 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.326934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.327087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.327175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.327339 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.327419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.327578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.327658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.327819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.327907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.328059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.328139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.328294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.328373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.328535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.328623 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.329068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.329154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.329302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.329381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.329537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.329632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.329776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.329857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.330008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.330089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.330245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.330333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.330478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.330558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.330716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.330849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.330978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.331114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.331241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.331365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.331500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.331626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.331752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.331885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.332011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.332136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.332136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.332371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.332497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.332631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.332757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.333044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.333659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.333790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.333930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.334027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.334159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.334280 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.334407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.334533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.334678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.334829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.334956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.335084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.335210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.335337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.335460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.335601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.335726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.335849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.335973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.336098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.336222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.336353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.336477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.336606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.336729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.336922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.337048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] resource.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.337186 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.337312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.337447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.337573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.337698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.337823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.337954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.338082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.338208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.338333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.338456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.338581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.338713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.338837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.338963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.339089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.339229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.339355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.339486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.339613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] role.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.339739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.339863 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] role.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.339986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.340183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.340312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.340437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.340561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.340685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.340824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.340958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.341100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.341229 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.341358 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] role.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.341485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.341630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] role.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.341759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.341888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.342016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.342143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.342270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.342397 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.342533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.342661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.342804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.342932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.343059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.343187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.343317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.343448 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.343575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.343701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.343828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.343955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.344089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.344217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.344345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.344483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.344611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.344738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.344909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.345039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.345167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.345295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.345422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.345561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.345699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.345827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.345953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.346079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.346215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.346349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.346482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.346608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.346738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.346864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.346988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.347123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.347248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.347373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.347500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.347626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.347753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.347894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.348020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.348146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.348271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.348395 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.348567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.348704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.348788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] token.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.348951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.349077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.349204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.349328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.349328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.349577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.349577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.349828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.349954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.350079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.350213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.350338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.350463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.350588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.350588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.350827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.350960 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.351085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.351210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.351335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.351468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.351595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.351733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.351862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.351987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.352111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.352237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.352361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.352496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.352631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.352718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.352794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.352884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.352957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.connection = **** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.353941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.354969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355684 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.355972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 10:41:38.356053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.356053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.356053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.356244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.356384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.356625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.356886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 10:41:38.357059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 10:41:38.384906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.385137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.385360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.385524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.385757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.385982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.393705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.394295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.422107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.422338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.422427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.422611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.476527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.476689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.525929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.526167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.526336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.526574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.683722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.683914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.684002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.684158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.684289 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.684399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.684482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.684561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.684639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.684776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.684900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.685023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.708452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.708822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.708921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.709060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.709163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.709282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.709367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.709500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.709583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.709714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.709798 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.709927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.710009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.710141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.721028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.721302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.734417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.734572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.734667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.734853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.734942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.735076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.735177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.735309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.735438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.735534 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.735612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.735716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.735800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.735934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 10:41:38.752690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.752891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.752980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 10:41:38.753105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 10:41:38.753265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.753452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.753552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 10:41:38.753657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 10:41:38.753741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.753935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.754021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.754199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.754282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.754460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.754542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.754717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.764105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.764369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.764456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.764650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.764734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.764948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.765046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.765238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.772446 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.772819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.775358 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.775649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.775732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.775939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.776022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.776227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.776308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.776514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.776598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.776777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.776929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.777075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.777172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.777350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.777433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.777623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.777703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.777892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.777974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.778162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.778242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.778435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.801344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.801461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.802919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.803046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.803201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.803336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.807860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.807998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.808146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.808284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.808431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.808568 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.814758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.814901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.815050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.815192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.820857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.820971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.821160 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.821245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.821455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.821539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.821747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.821829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.822029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.822113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.822308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.822398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.822582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.822664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.822853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.822934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.839067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.839181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.839349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.839433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.843155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.843248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.846358 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.846446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.846658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.846741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.851550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.851662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.854628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.854810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.857828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.858047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.858145 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.858351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.858435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.858643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.858726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.858933 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.859017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.859217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.859298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.859513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.874916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.875207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.875290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.875479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.875561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.875744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.875841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.876007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.883102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.883428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.884645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.884990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.886196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.886475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.886557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.886775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.889241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.889519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.889605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.889809 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.889891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.890091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.890187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.890386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.890474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.890776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.901715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.902004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.902089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.902291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.902375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.902574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.902665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.902861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.902945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.903229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.903311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.903516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.903600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.903808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.919582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.919683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.922506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.922599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.925418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.925504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.927020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.927155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.928497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.928649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.928784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.928956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.929201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.929356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.935533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.935644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.937011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.937114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.939884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.939973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.940282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.940368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.940571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.940656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.940871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.940961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.949184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.949406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.949496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.949706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.949794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.949994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.956843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.957088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 10:41:38.957185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.957373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.957460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.957657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.957743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.958490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.958598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.958784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.958990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.959080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.959157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.959339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.959442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.959618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.959704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.959900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.959986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.960179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.960264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.960460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.960589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.960723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.960840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.961015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.961098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.961290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 10:41:38.994849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555a92eed9c0 pid: 81387 (default app) Sep 15 10:41:38.994953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555a92eed9c0 pid: 81388 (default app) Sep 15 10:41:38.998052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50b832-c6c3-487f-8161-4e8dcd2c60a5 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:38.998341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50b832-c6c3-487f-8161-4e8dcd2c60a5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:38.998617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50b832-c6c3-487f-8161-4e8dcd2c60a5 None None] PATH_INFO: `/v3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:39.002755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1/1] 10.222.0.50 () {50 vars in 791 bytes} [Thu Sep 15 10:41:38 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:41:45.826449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36c7e7-cf9c-4219-800d-6a10051d34d0 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:45.827015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36c7e7-cf9c-4219-800d-6a10051d34d0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:45.827015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36c7e7-cf9c-4219-800d-6a10051d34d0 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:45.831434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1/2] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:41:45.836963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:45.837589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:45.837589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:45.908824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: INFO dbcounter [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] Registered counter for database keystone Sep 15 10:41:45.928894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG oslo_db.sqlalchemy.engines [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81387) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 10:41:45.935056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writer thread running {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 10:41:46.115971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG passlib.handlers.bcrypt [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81387) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 10:41:46.116279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG passlib.handlers.bcrypt [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81387) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 10:41:46.174682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:46.176314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:46.312344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 2/3] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 10:41:46.797733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36c7e7-cf9c-4219-800d-6a10051d34d0 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:46.869935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: INFO dbcounter [None req-6c36c7e7-cf9c-4219-800d-6a10051d34d0 None None] Registered counter for database keystone Sep 15 10:41:46.891677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG oslo_db.sqlalchemy.engines [None req-6c36c7e7-cf9c-4219-800d-6a10051d34d0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81388) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 10:41:46.898152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writer thread running {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 10:41:47.018284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36c7e7-cf9c-4219-800d-6a10051d34d0 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:47.022715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee640b44-62bd-4fcc-8c56-fbf0b684788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:47.023338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee640b44-62bd-4fcc-8c56-fbf0b684788c admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:47.023620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee640b44-62bd-4fcc-8c56-fbf0b684788c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:47.023893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee640b44-62bd-4fcc-8c56-fbf0b684788c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:47.032188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee640b44-62bd-4fcc-8c56-fbf0b684788c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:47.085873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ee640b44-62bd-4fcc-8c56-fbf0b684788c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 10:41:47.086214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee640b44-62bd-4fcc-8c56-fbf0b684788c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:47.095656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 2/4] 10.222.0.50 () {58 vars in 1324 bytes} [Thu Sep 15 10:41:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 10:41:47.102184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:47.120677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d0b03-321d-4255-9c67-1acb02c3e155 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:47.125015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719f55-faa9-4875-8fb6-6dd6e8322c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:47.126133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719f55-faa9-4875-8fb6-6dd6e8322c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:47.126409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719f55-faa9-4875-8fb6-6dd6e8322c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:47.126673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719f55-faa9-4875-8fb6-6dd6e8322c2a admin admin] PATH_INFO: `/v3/projects/70f07107ef09454996be197ead3ddf41` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:47.128265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f719f55-faa9-4875-8fb6-6dd6e8322c2a admin admin] RBAC: Authorizing `identity:get_project(project_id=70f07107ef09454996be197ead3ddf41)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:47.183464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2f719f55-faa9-4875-8fb6-6dd6e8322c2a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 10:41:47.183808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f719f55-faa9-4875-8fb6-6dd6e8322c2a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:47.188296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 3/5] 10.222.0.50 () {56 vars in 1207 bytes} [Thu Sep 15 10:41:47 2022] GET /identity/v3/projects/70f07107ef09454996be197ead3ddf41 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:41:49.315409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366063-f258-491d-966e-46b775612968 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:49.315965 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366063-f258-491d-966e-46b775612968 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:49.315965 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366063-f258-491d-966e-46b775612968 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:49.320839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 3/6] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:41:49.326228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3d6db-20dc-4370-b0cc-a4c3785dd0d2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:49.326615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3d6db-20dc-4370-b0cc-a4c3785dd0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:49.326791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3d6db-20dc-4370-b0cc-a4c3785dd0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:49.347382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dee3d6db-20dc-4370-b0cc-a4c3785dd0d2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:49.349051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dee3d6db-20dc-4370-b0cc-a4c3785dd0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:49.356837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 4/7] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 10:41:49.627193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5366063-f258-491d-966e-46b775612968 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:49.639739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5366063-f258-491d-966e-46b775612968 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:49.643984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb7366-0aa3-468a-b469-4b00296c85bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:49.644584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb7366-0aa3-468a-b469-4b00296c85bd admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:49.644934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb7366-0aa3-468a-b469-4b00296c85bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:49.645302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb7366-0aa3-468a-b469-4b00296c85bd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:49.653134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb7366-0aa3-468a-b469-4b00296c85bd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:49.654927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb7366-0aa3-468a-b469-4b00296c85bd admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:49.664299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 4/8] 10.222.0.50 () {58 vars in 1324 bytes} [Thu Sep 15 10:41:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 10:41:49.669631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3d6db-20dc-4370-b0cc-a4c3785dd0d2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:49.679295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3d6db-20dc-4370-b0cc-a4c3785dd0d2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:49.683622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca46fb-f5df-40f0-a4ad-2d40502f92aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:49.684246 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca46fb-f5df-40f0-a4ad-2d40502f92aa admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:49.684525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca46fb-f5df-40f0-a4ad-2d40502f92aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:49.684794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca46fb-f5df-40f0-a4ad-2d40502f92aa admin admin] PATH_INFO: `/v3/users/c62c262317454775845cb45d452c8ab2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:49.686121 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca46fb-f5df-40f0-a4ad-2d40502f92aa admin admin] RBAC: Authorizing `identity:get_user(user_id=c62c262317454775845cb45d452c8ab2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:49.687964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca46fb-f5df-40f0-a4ad-2d40502f92aa admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:49.689521 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 5/9] 10.222.0.50 () {56 vars in 1198 bytes} [Thu Sep 15 10:41:49 2022] GET /identity/v3/users/c62c262317454775845cb45d452c8ab2 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:41:51.914349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3221859-6066-4841-9da1-33b54c91b619 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:51.914849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3221859-6066-4841-9da1-33b54c91b619 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:51.914932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3221859-6066-4841-9da1-33b54c91b619 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:51.916231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 5/10] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:41:51.921378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c4d45-733f-46c4-8a1b-99644a4b0bbf None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:51.921769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c4d45-733f-46c4-8a1b-99644a4b0bbf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:51.922057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c4d45-733f-46c4-8a1b-99644a4b0bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:51.929299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9370559-f988-4b88-92ab-71b6ba01fa13 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:51.929585 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9370559-f988-4b88-92ab-71b6ba01fa13 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:51.929855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9370559-f988-4b88-92ab-71b6ba01fa13 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:51.931160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 6/11] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:41:51.936361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8db5de-faed-46b8-9a62-346b650415ac None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:51.936670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8db5de-faed-46b8-9a62-346b650415ac None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:51.936974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8db5de-faed-46b8-9a62-346b650415ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:51.941090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-858c4d45-733f-46c4-8a1b-99644a4b0bbf None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:51.942342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-858c4d45-733f-46c4-8a1b-99644a4b0bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:51.979422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG passlib.handlers.bcrypt [None req-5d8db5de-faed-46b8-9a62-346b650415ac None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81388) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 10:41:51.979807 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG passlib.handlers.bcrypt [None req-5d8db5de-faed-46b8-9a62-346b650415ac None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81388) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 10:41:52.023724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 6/12] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:41:52.030704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5d8db5de-faed-46b8-9a62-346b650415ac None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:52.032314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5d8db5de-faed-46b8-9a62-346b650415ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:52.133952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 7/13] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:41:52.294222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c4d45-733f-46c4-8a1b-99644a4b0bbf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:52.319900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:52.399655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c4d45-733f-46c4-8a1b-99644a4b0bbf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:52.403825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8db5de-faed-46b8-9a62-346b650415ac None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:52.433476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab4489-2ff2-4929-9298-a53a2d1cb897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:52.434115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab4489-2ff2-4929-9298-a53a2d1cb897 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:52.434400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab4489-2ff2-4929-9298-a53a2d1cb897 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:52.434676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab4489-2ff2-4929-9298-a53a2d1cb897 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:52.446172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab4489-2ff2-4929-9298-a53a2d1cb897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-26ab4489-2ff2-4929-9298-a53a2d1cb897 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:41:52.455317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:41:52.456386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:41:52.458341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 7/14] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:41:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:41:52.462868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab4489-2ff2-4929-9298-a53a2d1cb897 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:52.506751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8db5de-faed-46b8-9a62-346b650415ac None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:52.509099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:52.544968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:52.545951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a04e2-ce05-465f-991c-91d96ab75ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:52.546781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a04e2-ce05-465f-991c-91d96ab75ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:52.547023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a04e2-ce05-465f-991c-91d96ab75ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:52.547345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a04e2-ce05-465f-991c-91d96ab75ba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:52.564706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab4489-2ff2-4929-9298-a53a2d1cb897 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:52.599365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df15d4-f21c-42f1-9093-19fa1ed90177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:52.599976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df15d4-f21c-42f1-9093-19fa1ed90177 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:52.600265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df15d4-f21c-42f1-9093-19fa1ed90177 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:52.600543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df15d4-f21c-42f1-9093-19fa1ed90177 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:52.601238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df15d4-f21c-42f1-9093-19fa1ed90177 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:41:52.601542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df15d4-f21c-42f1-9093-19fa1ed90177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:52.602950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df15d4-f21c-42f1-9093-19fa1ed90177 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:52.614636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 8/15] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:41:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:41:52.615438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a04e2-ce05-465f-991c-91d96ab75ba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:52.617038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a04e2-ce05-465f-991c-91d96ab75ba5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:52.620071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df15d4-f21c-42f1-9093-19fa1ed90177 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:52.718360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 8/16] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 10:41:52.719565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df15d4-f21c-42f1-9093-19fa1ed90177 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:52.724450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a04e2-ce05-465f-991c-91d96ab75ba5 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:52.727180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:52.754275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159124c-eb4d-4d96-b098-e8dbf99fa176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:52.754916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159124c-eb4d-4d96-b098-e8dbf99fa176 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:52.755201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159124c-eb4d-4d96-b098-e8dbf99fa176 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:52.755474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159124c-eb4d-4d96-b098-e8dbf99fa176 None admin] PATH_INFO: `/v3/users/c62c262317454775845cb45d452c8ab2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:52.756619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159124c-eb4d-4d96-b098-e8dbf99fa176 None admin] RBAC: Authorizing `identity:get_user(user_id=c62c262317454775845cb45d452c8ab2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:52.758117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159124c-eb4d-4d96-b098-e8dbf99fa176 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:52.759702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 9/17] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:41:52 2022] GET /identity/v3/users/c62c262317454775845cb45d452c8ab2 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:41:52.765023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159124c-eb4d-4d96-b098-e8dbf99fa176 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:52.784917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:52.823885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a04e2-ce05-465f-991c-91d96ab75ba5 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:52.859068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40989b67-dbd1-4265-b07d-cb1a93cfee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:52.859925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40989b67-dbd1-4265-b07d-cb1a93cfee9c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:52.860208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40989b67-dbd1-4265-b07d-cb1a93cfee9c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:52.860487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40989b67-dbd1-4265-b07d-cb1a93cfee9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:52.871288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159124c-eb4d-4d96-b098-e8dbf99fa176 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:52.878685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40989b67-dbd1-4265-b07d-cb1a93cfee9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:52.880236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40989b67-dbd1-4265-b07d-cb1a93cfee9c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-40989b67-dbd1-4265-b07d-cb1a93cfee9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:41:52.898872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:41:52.899759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:41:52.900711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:41:52.900711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:41:52.900711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:41:52.900711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 9/18] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:41:52.904350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa603f-5cf9-43cf-ad03-1c560df7c46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:52.905022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa603f-5cf9-43cf-ad03-1c560df7c46a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:52.905302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa603f-5cf9-43cf-ad03-1c560df7c46a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:52.905577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa603f-5cf9-43cf-ad03-1c560df7c46a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:52.905767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40989b67-dbd1-4265-b07d-cb1a93cfee9c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:52.906687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa603f-5cf9-43cf-ad03-1c560df7c46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:52.908148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa603f-5cf9-43cf-ad03-1c560df7c46a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:52.909873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 10/19] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:41:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:41:52.916135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa603f-5cf9-43cf-ad03-1c560df7c46a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:52.962405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:52.994913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:53.008763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40989b67-dbd1-4265-b07d-cb1a93cfee9c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:53.015053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa603f-5cf9-43cf-ad03-1c560df7c46a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:53.042872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14038e85-4d73-412e-9868-da1eb7c97102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:53.043489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14038e85-4d73-412e-9868-da1eb7c97102 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:53.043770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14038e85-4d73-412e-9868-da1eb7c97102 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:53.044049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14038e85-4d73-412e-9868-da1eb7c97102 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:53.044698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14038e85-4d73-412e-9868-da1eb7c97102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:41:53.045036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14038e85-4d73-412e-9868-da1eb7c97102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:53.046433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14038e85-4d73-412e-9868-da1eb7c97102 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:53.048626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eaf2cd-c1db-44d6-9654-d9d89aa1a650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:53.049273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eaf2cd-c1db-44d6-9654-d9d89aa1a650 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:53.049561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eaf2cd-c1db-44d6-9654-d9d89aa1a650 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:53.049835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eaf2cd-c1db-44d6-9654-d9d89aa1a650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:53.050560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eaf2cd-c1db-44d6-9654-d9d89aa1a650 None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.domain.id=default, user.id=c62c262317454775845cb45d452c8ab2) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:41:53.050869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eaf2cd-c1db-44d6-9654-d9d89aa1a650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:53.053003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eaf2cd-c1db-44d6-9654-d9d89aa1a650 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:53.059556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 11/20] 10.222.0.50 () {56 vars in 1322 bytes} [Thu Sep 15 10:41:52 2022] GET /identity/v3/role_assignments?user.id=c62c262317454775845cb45d452c8ab2&scope.domain.id=default&role.id=0be9777002c641b4a47f78df647cc173 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:41:53.064107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 10/21] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:41:55.455171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9385f-618a-41d5-8b8e-96ac88af7426 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:55.455725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9385f-618a-41d5-8b8e-96ac88af7426 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:55.455725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9385f-618a-41d5-8b8e-96ac88af7426 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:55.457126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 12/22] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:41:55.468195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954e4d-969e-44c0-8446-db0171c02206 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:55.468698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954e4d-969e-44c0-8446-db0171c02206 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:55.468895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954e4d-969e-44c0-8446-db0171c02206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:55.498287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ed954e4d-969e-44c0-8446-db0171c02206 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:55.499500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ed954e4d-969e-44c0-8446-db0171c02206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:55.592098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1c8f9-72e8-41ab-8e63-2267944f2f49 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:55.592415 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1c8f9-72e8-41ab-8e63-2267944f2f49 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:55.592653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1c8f9-72e8-41ab-8e63-2267944f2f49 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:55.594183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 13/23] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:41:55.600513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250847c1-ac74-4756-a72c-8cc7b800b77c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:55.601125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250847c1-ac74-4756-a72c-8cc7b800b77c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:55.601603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250847c1-ac74-4756-a72c-8cc7b800b77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:55.614125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 11/24] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:41:55.637303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-250847c1-ac74-4756-a72c-8cc7b800b77c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:55.638512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-250847c1-ac74-4756-a72c-8cc7b800b77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:55.737572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 14/25] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:41:55.917961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fea769-d626-4549-9d18-4d74e145e268 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:55.918339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fea769-d626-4549-9d18-4d74e145e268 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:55.918533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fea769-d626-4549-9d18-4d74e145e268 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:55.920128 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 12/26] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:41:55.925631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cff2a-662f-43c0-9c9c-726c4f3c4ad3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:55.925957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cff2a-662f-43c0-9c9c-726c4f3c4ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:55.926327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cff2a-662f-43c0-9c9c-726c4f3c4ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:55.949559 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7f1cff2a-662f-43c0-9c9c-726c4f3c4ad3 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:55.950747 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7f1cff2a-662f-43c0-9c9c-726c4f3c4ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:56.027399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 15/27] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:41:56.200848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fea769-d626-4549-9d18-4d74e145e268 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:56.261820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:56.334379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cff2a-662f-43c0-9c9c-726c4f3c4ad3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:56.345129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:56.351680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fea769-d626-4549-9d18-4d74e145e268 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:56.431504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1932503-b062-4231-9f2e-0cd783bb82df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:56.432155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1932503-b062-4231-9f2e-0cd783bb82df None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:56.432444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1932503-b062-4231-9f2e-0cd783bb82df None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:56.432707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1932503-b062-4231-9f2e-0cd783bb82df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:56.455772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1932503-b062-4231-9f2e-0cd783bb82df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:56.457872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1932503-b062-4231-9f2e-0cd783bb82df None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-e1932503-b062-4231-9f2e-0cd783bb82df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:41:56.474985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:41:56.476299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:41:56.477551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:41:56.477551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:41:56.477551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:41:56.477551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 13/28] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:41:56.477758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1932503-b062-4231-9f2e-0cd783bb82df None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:56.534636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cff2a-662f-43c0-9c9c-726c4f3c4ad3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:56.579021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330984a-4f03-4ecb-b24e-5dc34ba7d7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:56.579882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330984a-4f03-4ecb-b24e-5dc34ba7d7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:56.580348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330984a-4f03-4ecb-b24e-5dc34ba7d7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:56.580767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330984a-4f03-4ecb-b24e-5dc34ba7d7c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:56.592495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330984a-4f03-4ecb-b24e-5dc34ba7d7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-4330984a-4f03-4ecb-b24e-5dc34ba7d7c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:41:56.598143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:41:56.599209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:41:56.600030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:41:56.602053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 16/29] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:41:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:41:56.603116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330984a-4f03-4ecb-b24e-5dc34ba7d7c7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:56.607840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:56.612426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1932503-b062-4231-9f2e-0cd783bb82df None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:56.657898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6b2a5-69db-43b2-80f2-85a5b0b1195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:56.658672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6b2a5-69db-43b2-80f2-85a5b0b1195a None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:56.659063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6b2a5-69db-43b2-80f2-85a5b0b1195a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:56.659687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6b2a5-69db-43b2-80f2-85a5b0b1195a None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:56.660927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6b2a5-69db-43b2-80f2-85a5b0b1195a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:56.665209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6b2a5-69db-43b2-80f2-85a5b0b1195a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:56.677022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:56.724623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330984a-4f03-4ecb-b24e-5dc34ba7d7c7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:56.762773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc649f-a171-4604-aac1-d3c24f26a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:56.763534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc649f-a171-4604-aac1-d3c24f26a491 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:56.763907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc649f-a171-4604-aac1-d3c24f26a491 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:56.764274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc649f-a171-4604-aac1-d3c24f26a491 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:56.765050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc649f-a171-4604-aac1-d3c24f26a491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:41:56.765450 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc649f-a171-4604-aac1-d3c24f26a491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:56.766935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc649f-a171-4604-aac1-d3c24f26a491 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:56.787495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 17/30] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:41:56.789355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387bf2b-795d-47b6-90b3-b841132ef2f6 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:56.789787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387bf2b-795d-47b6-90b3-b841132ef2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:56.790200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387bf2b-795d-47b6-90b3-b841132ef2f6 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:56.795978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 18/31] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:41:56.795978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387bf2b-795d-47b6-90b3-b841132ef2f6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:56.820244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 14/32] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Sep 15 10:41:56 2022] POST /identity/v3/roles => generated 219 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:41:56.821859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6b2a5-69db-43b2-80f2-85a5b0b1195a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:56.963272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387bf2b-795d-47b6-90b3-b841132ef2f6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:57.006580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:57.027514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6b2a5-69db-43b2-80f2-85a5b0b1195a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:57.031258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e1e4a-1d43-4738-b658-19083a73be36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:57.031882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e1e4a-1d43-4738-b658-19083a73be36 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.032159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e1e4a-1d43-4738-b658-19083a73be36 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.032427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e1e4a-1d43-4738-b658-19083a73be36 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.033120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e1e4a-1d43-4738-b658-19083a73be36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:41:57.033422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e1e4a-1d43-4738-b658-19083a73be36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:57.037536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e1e4a-1d43-4738-b658-19083a73be36 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:57.053908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 19/33] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:41:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:41:57.055182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b14ac5-9674-426c-9edb-f9f00770f5a9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.055474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b14ac5-9674-426c-9edb-f9f00770f5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.055749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b14ac5-9674-426c-9edb-f9f00770f5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.079757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-67b14ac5-9674-426c-9edb-f9f00770f5a9 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:57.081010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-67b14ac5-9674-426c-9edb-f9f00770f5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:57.081907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe74d5e-6c80-43de-a1b6-1444d385d355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:57.082855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe74d5e-6c80-43de-a1b6-1444d385d355 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.083498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe74d5e-6c80-43de-a1b6-1444d385d355 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.083788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe74d5e-6c80-43de-a1b6-1444d385d355 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.085837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe74d5e-6c80-43de-a1b6-1444d385d355 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:57.087814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe74d5e-6c80-43de-a1b6-1444d385d355 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:57.137921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 15/34] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:41:56 2022] POST /identity/v3/projects => generated 301 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:41:57.138812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe74d5e-6c80-43de-a1b6-1444d385d355 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:57.167362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:57.203159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 20/35] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:41:57.314359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe74d5e-6c80-43de-a1b6-1444d385d355 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:57.358909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b13d3-f99e-476e-b2af-4687bacd71b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:57.359572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b13d3-f99e-476e-b2af-4687bacd71b7 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.359854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b13d3-f99e-476e-b2af-4687bacd71b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.360172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b13d3-f99e-476e-b2af-4687bacd71b7 None admin] PATH_INFO: `/v3/users/c62c262317454775845cb45d452c8ab2` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.361639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b13d3-f99e-476e-b2af-4687bacd71b7 None admin] RBAC: Authorizing `identity:get_user(user_id=c62c262317454775845cb45d452c8ab2)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:57.363184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b13d3-f99e-476e-b2af-4687bacd71b7 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:57.365770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 16/36] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:41:57 2022] GET /identity/v3/users/c62c262317454775845cb45d452c8ab2 => generated 247 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:41:57.369918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f6f69-a8b9-4603-a3ac-d26ecb67f9fd None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.370591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f6f69-a8b9-4603-a3ac-d26ecb67f9fd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.371005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f6f69-a8b9-4603-a3ac-d26ecb67f9fd None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.371160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b13d3-f99e-476e-b2af-4687bacd71b7 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:57.374289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 21/37] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:41:57.378703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1ca62-68fa-4744-902f-16c8f718e3b0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.379244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1ca62-68fa-4744-902f-16c8f718e3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.379991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1ca62-68fa-4744-902f-16c8f718e3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.401870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d3b1ca62-68fa-4744-902f-16c8f718e3b0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:57.404556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d3b1ca62-68fa-4744-902f-16c8f718e3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:57.471530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:57.492972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 22/38] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:41:57.492972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef35aa5-7813-4a61-b6af-b7b6c14d4a42 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.492972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef35aa5-7813-4a61-b6af-b7b6c14d4a42 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.492972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef35aa5-7813-4a61-b6af-b7b6c14d4a42 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.493498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 23/39] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:41:57.497805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b13d3-f99e-476e-b2af-4687bacd71b7 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:57.498731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c83f8-242e-460d-b348-5afcf70b3f3f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.499020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c83f8-242e-460d-b348-5afcf70b3f3f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.499298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c83f8-242e-460d-b348-5afcf70b3f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.518454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-376c83f8-242e-460d-b348-5afcf70b3f3f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:41:57.519870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-376c83f8-242e-460d-b348-5afcf70b3f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:41:57.535621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263143fc-9fc4-4394-b00d-2ae1ed7d5cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:57.536273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263143fc-9fc4-4394-b00d-2ae1ed7d5cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.536561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263143fc-9fc4-4394-b00d-2ae1ed7d5cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.536865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263143fc-9fc4-4394-b00d-2ae1ed7d5cc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.538033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263143fc-9fc4-4394-b00d-2ae1ed7d5cc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:57.539522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263143fc-9fc4-4394-b00d-2ae1ed7d5cc9 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:57.541246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 17/40] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:41:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:41:57.541995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263143fc-9fc4-4394-b00d-2ae1ed7d5cc9 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:57.580622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:57.608634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 24/41] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:41:57.609733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c83f8-242e-460d-b348-5afcf70b3f3f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:57.666749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263143fc-9fc4-4394-b00d-2ae1ed7d5cc9 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:57.707894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b572c4-bc80-41ca-9647-07d042781da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:57.709440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b572c4-bc80-41ca-9647-07d042781da6 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.710145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b572c4-bc80-41ca-9647-07d042781da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.710587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b572c4-bc80-41ca-9647-07d042781da6 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.711347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b572c4-bc80-41ca-9647-07d042781da6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:57.713062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b572c4-bc80-41ca-9647-07d042781da6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:57.730067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 18/42] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Sep 15 10:41:57 2022] POST /identity/v3/roles => generated 225 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:41:57.737337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c83f8-242e-460d-b348-5afcf70b3f3f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:57.771872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b572c4-bc80-41ca-9647-07d042781da6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:57.786980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb041b0-065e-4949-b8dc-1e5a2628d3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:57.787733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:57.787843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb041b0-065e-4949-b8dc-1e5a2628d3bb None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.788111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb041b0-065e-4949-b8dc-1e5a2628d3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.788414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb041b0-065e-4949-b8dc-1e5a2628d3bb None admin] PATH_INFO: `/v3/domains/default/users/c62c262317454775845cb45d452c8ab2/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.790595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb041b0-065e-4949-b8dc-1e5a2628d3bb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c62c262317454775845cb45d452c8ab2, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:57.792223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb041b0-065e-4949-b8dc-1e5a2628d3bb None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:57.905281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b572c4-bc80-41ca-9647-07d042781da6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:57.951941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b37cb-e210-46bb-a03d-64de227c75fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:57.952602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b37cb-e210-46bb-a03d-64de227c75fd None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:57.952945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b37cb-e210-46bb-a03d-64de227c75fd None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:57.953223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b37cb-e210-46bb-a03d-64de227c75fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:57.954711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b37cb-e210-46bb-a03d-64de227c75fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:57.956232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b37cb-e210-46bb-a03d-64de227c75fd None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:57.959215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 19/43] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:41:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:41:57.959906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b37cb-e210-46bb-a03d-64de227c75fd None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:58.039904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:58.044071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 25/44] 10.222.0.50 () {58 vars in 1361 bytes} [Thu Sep 15 10:41:57 2022] PUT /identity/v3/domains/default/users/c62c262317454775845cb45d452c8ab2/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:41:58.045312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb041b0-065e-4949-b8dc-1e5a2628d3bb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:41:58.069849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b37cb-e210-46bb-a03d-64de227c75fd None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:58.086234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:41:58.116556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3277906-b5d2-4f72-bb19-fe9c60937214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:58.117340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3277906-b5d2-4f72-bb19-fe9c60937214 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:58.117632 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3277906-b5d2-4f72-bb19-fe9c60937214 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:58.117909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3277906-b5d2-4f72-bb19-fe9c60937214 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:58.118685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3277906-b5d2-4f72-bb19-fe9c60937214 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:58.120174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3277906-b5d2-4f72-bb19-fe9c60937214 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:58.145154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 20/45] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Sep 15 10:41:57 2022] POST /identity/v3/roles => generated 223 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:41:58.166180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb041b0-065e-4949-b8dc-1e5a2628d3bb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:41:58.209630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f49ea3-d37d-404e-aa65-0524f19732d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:41:58.210297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f49ea3-d37d-404e-aa65-0524f19732d6 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:41:58.210582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f49ea3-d37d-404e-aa65-0524f19732d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:41:58.210859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f49ea3-d37d-404e-aa65-0524f19732d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:41:58.211662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f49ea3-d37d-404e-aa65-0524f19732d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:41:58.213516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f49ea3-d37d-404e-aa65-0524f19732d6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:41:58.252051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 26/46] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:41:58 2022] POST /identity/v3/projects => generated 312 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:42:00.231057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334daa5-1df0-495c-b35d-4998ba326678 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:00.231578 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334daa5-1df0-495c-b35d-4998ba326678 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:00.231691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334daa5-1df0-495c-b35d-4998ba326678 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:00.233083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 21/47] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:00.238579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818f207-a53f-4d60-8c40-95d161fcd722 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:00.239491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818f207-a53f-4d60-8c40-95d161fcd722 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:00.239740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818f207-a53f-4d60-8c40-95d161fcd722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:00.257855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c818f207-a53f-4d60-8c40-95d161fcd722 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:00.259385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c818f207-a53f-4d60-8c40-95d161fcd722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:00.328300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 27/48] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:00.422306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb6c46-08be-41c6-811e-5fa149bec91d None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:00.422595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb6c46-08be-41c6-811e-5fa149bec91d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:00.422877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb6c46-08be-41c6-811e-5fa149bec91d None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:00.424225 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 22/49] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:00.430401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717845e-1a93-44d4-a8fc-724f255b198c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:00.430717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717845e-1a93-44d4-a8fc-724f255b198c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:00.431048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717845e-1a93-44d4-a8fc-724f255b198c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:00.450563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2717845e-1a93-44d4-a8fc-724f255b198c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:00.451786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2717845e-1a93-44d4-a8fc-724f255b198c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:00.465123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:00.520402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 28/50] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:00.600915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb6c46-08be-41c6-811e-5fa149bec91d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:00.704958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb6c46-08be-41c6-811e-5fa149bec91d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:00.737847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc949bf7-fe58-4bdc-bd18-fd12b9704929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:00.739192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc949bf7-fe58-4bdc-bd18-fd12b9704929 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:00.739474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc949bf7-fe58-4bdc-bd18-fd12b9704929 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:00.739754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc949bf7-fe58-4bdc-bd18-fd12b9704929 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:00.741771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:00.753165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc949bf7-fe58-4bdc-bd18-fd12b9704929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-dc949bf7-fe58-4bdc-bd18-fd12b9704929 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:00.759777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:00.760841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:00.761673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 23/51] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:42:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:00.766727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717845e-1a93-44d4-a8fc-724f255b198c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:00.795615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc949bf7-fe58-4bdc-bd18-fd12b9704929 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:00.867027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717845e-1a93-44d4-a8fc-724f255b198c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:00.896875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc949bf7-fe58-4bdc-bd18-fd12b9704929 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:00.901567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e56c4-1538-4015-9cd8-6e38ecbbe3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:00.902184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e56c4-1538-4015-9cd8-6e38ecbbe3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:00.902470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e56c4-1538-4015-9cd8-6e38ecbbe3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:00.902751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e56c4-1538-4015-9cd8-6e38ecbbe3e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:00.903424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e56c4-1538-4015-9cd8-6e38ecbbe3e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:00.903740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e56c4-1538-4015-9cd8-6e38ecbbe3e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:00.905178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e56c4-1538-4015-9cd8-6e38ecbbe3e0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:00.913988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 29/52] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:00.919603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e56c4-1538-4015-9cd8-6e38ecbbe3e0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:00.930990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be75be-6c11-4663-950b-e248d2626e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:00.931622 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be75be-6c11-4663-950b-e248d2626e7c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:00.931903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be75be-6c11-4663-950b-e248d2626e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:00.932180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be75be-6c11-4663-950b-e248d2626e7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:00.932299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:00.933562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be75be-6c11-4663-950b-e248d2626e7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:00.935036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be75be-6c11-4663-950b-e248d2626e7c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:00.936595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 24/53] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:00.942253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be75be-6c11-4663-950b-e248d2626e7c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:01.004876 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:01.022626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e56c4-1538-4015-9cd8-6e38ecbbe3e0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:01.048726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be75be-6c11-4663-950b-e248d2626e7c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:01.058211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f991368-e659-4d93-be66-68b0a7925447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:01.059170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f991368-e659-4d93-be66-68b0a7925447 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:01.059465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f991368-e659-4d93-be66-68b0a7925447 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:01.059748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f991368-e659-4d93-be66-68b0a7925447 None admin] PATH_INFO: `/v3/users/c62c262317454775845cb45d452c8ab2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:01.060940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f991368-e659-4d93-be66-68b0a7925447 None admin] RBAC: Authorizing `identity:get_user(user_id=c62c262317454775845cb45d452c8ab2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:01.062420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f991368-e659-4d93-be66-68b0a7925447 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:01.064018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 30/54] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:00 2022] GET /identity/v3/users/c62c262317454775845cb45d452c8ab2 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:01.069590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f991368-e659-4d93-be66-68b0a7925447 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:01.083509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f27076-8f34-4be8-9a82-88e98963c991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:01.084124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f27076-8f34-4be8-9a82-88e98963c991 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:01.084422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f27076-8f34-4be8-9a82-88e98963c991 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:01.084684 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f27076-8f34-4be8-9a82-88e98963c991 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:01.085368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f27076-8f34-4be8-9a82-88e98963c991 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:01.086815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f27076-8f34-4be8-9a82-88e98963c991 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:01.113783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 25/55] 10.222.0.50 () {60 vars in 1140 bytes} [Thu Sep 15 10:42:00 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:42:01.150951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:01.175083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f991368-e659-4d93-be66-68b0a7925447 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:01.214407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26394397-c9ce-4eba-8877-1b2aa9256535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:01.215094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26394397-c9ce-4eba-8877-1b2aa9256535 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:01.215383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26394397-c9ce-4eba-8877-1b2aa9256535 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:01.215659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26394397-c9ce-4eba-8877-1b2aa9256535 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:01.216898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26394397-c9ce-4eba-8877-1b2aa9256535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:01.218400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26394397-c9ce-4eba-8877-1b2aa9256535 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:01.220021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 31/56] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:01.227106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f27076-8f34-4be8-9a82-88e98963c991 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:01.331524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f27076-8f34-4be8-9a82-88e98963c991 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:01.347337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:01.368284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9601210-d24e-41a8-8526-220ba370b9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:01.368920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9601210-d24e-41a8-8526-220ba370b9ea None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:01.369205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9601210-d24e-41a8-8526-220ba370b9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:01.369481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9601210-d24e-41a8-8526-220ba370b9ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:01.370206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9601210-d24e-41a8-8526-220ba370b9ea None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.domain.id=default, user.id=c62c262317454775845cb45d452c8ab2) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:01.370521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9601210-d24e-41a8-8526-220ba370b9ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:01.371984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9601210-d24e-41a8-8526-220ba370b9ea None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:01.381105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 26/57] 10.222.0.50 () {56 vars in 1322 bytes} [Thu Sep 15 10:42:01 2022] GET /identity/v3/role_assignments?user.id=c62c262317454775845cb45d452c8ab2&scope.domain.id=default&role.id=0be9777002c641b4a47f78df647cc173 => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:03.284667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cff74-5f45-42b6-b56e-f49d00a35186 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:03.285416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cff74-5f45-42b6-b56e-f49d00a35186 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:03.285416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cff74-5f45-42b6-b56e-f49d00a35186 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:03.286708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 32/58] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:03.291917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1892ba-f9d1-4b2a-9d30-c7c91c8d234f None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:03.292334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1892ba-f9d1-4b2a-9d30-c7c91c8d234f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:03.292482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1892ba-f9d1-4b2a-9d30-c7c91c8d234f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:03.412832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-dd1892ba-f9d1-4b2a-9d30-c7c91c8d234f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:03.414385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-dd1892ba-f9d1-4b2a-9d30-c7c91c8d234f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:03.479504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 27/59] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:03.746615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cff74-5f45-42b6-b56e-f49d00a35186 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:03.846550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cff74-5f45-42b6-b56e-f49d00a35186 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:03.878330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c01f42-7b11-4e84-8561-189eab393744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:03.879652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c01f42-7b11-4e84-8561-189eab393744 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:03.879984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c01f42-7b11-4e84-8561-189eab393744 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:03.880260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c01f42-7b11-4e84-8561-189eab393744 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:03.881374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c01f42-7b11-4e84-8561-189eab393744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:03.882835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c01f42-7b11-4e84-8561-189eab393744 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:03.884375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 33/60] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:03.889740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1892ba-f9d1-4b2a-9d30-c7c91c8d234f None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:03.984592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1892ba-f9d1-4b2a-9d30-c7c91c8d234f None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:03.993436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:04.017980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ed2b4-34be-49e2-b50e-5b486c821e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:04.018623 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ed2b4-34be-49e2-b50e-5b486c821e45 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:04.018911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ed2b4-34be-49e2-b50e-5b486c821e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:04.019191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ed2b4-34be-49e2-b50e-5b486c821e45 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:04.020439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ed2b4-34be-49e2-b50e-5b486c821e45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:04.021928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ed2b4-34be-49e2-b50e-5b486c821e45 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:04.047702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 28/61] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:42:03 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:42:07.042619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cde74-d174-4d8e-91f4-d33a7486c0bf None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:07.043671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cde74-d174-4d8e-91f4-d33a7486c0bf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:07.044188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cde74-d174-4d8e-91f4-d33a7486c0bf None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:07.045797 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 34/62] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:07.051100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0e403-3386-43bf-8c1d-16d9651b0e85 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:07.051913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0e403-3386-43bf-8c1d-16d9651b0e85 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:07.052430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0e403-3386-43bf-8c1d-16d9651b0e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:07.082363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-32c0e403-3386-43bf-8c1d-16d9651b0e85 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:07.084106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-32c0e403-3386-43bf-8c1d-16d9651b0e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:07.174118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 29/63] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:07.529438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cde74-d174-4d8e-91f4-d33a7486c0bf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:07.552785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:07.676208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daf71b-27d4-4e0a-8ed9-efaa867a254c None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:07.676887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daf71b-27d4-4e0a-8ed9-efaa867a254c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:07.677398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daf71b-27d4-4e0a-8ed9-efaa867a254c None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:07.678977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 30/64] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:07.684739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec7281-6146-4e16-940c-7727a0a9987d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:07.684997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cde74-d174-4d8e-91f4-d33a7486c0bf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:07.685516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec7281-6146-4e16-940c-7727a0a9987d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:07.686067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec7281-6146-4e16-940c-7727a0a9987d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:07.705036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ffec7281-6146-4e16-940c-7727a0a9987d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:07.706581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ffec7281-6146-4e16-940c-7727a0a9987d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:07.734407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff07253-f09d-4fb0-b8d3-89b4a15cdebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:07.735099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff07253-f09d-4fb0-b8d3-89b4a15cdebb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:07.735324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff07253-f09d-4fb0-b8d3-89b4a15cdebb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:07.735598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff07253-f09d-4fb0-b8d3-89b4a15cdebb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:07.749188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff07253-f09d-4fb0-b8d3-89b4a15cdebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-cff07253-f09d-4fb0-b8d3-89b4a15cdebb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:07.755378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:07.756288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:07.757120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:07.761476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 35/65] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:07.762794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897ef63-0a87-41e1-bc02-2650455ada0d None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:07.763086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897ef63-0a87-41e1-bc02-2650455ada0d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:07.763367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897ef63-0a87-41e1-bc02-2650455ada0d None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:07.765144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 36/66] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:07.765853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897ef63-0a87-41e1-bc02-2650455ada0d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:07.787464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 31/67] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:07.789172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fea69c-1c3e-48a9-aa0c-67ffd0747b39 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:07.789688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fea69c-1c3e-48a9-aa0c-67ffd0747b39 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:07.790164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fea69c-1c3e-48a9-aa0c-67ffd0747b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:07.809296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-36fea69c-1c3e-48a9-aa0c-67ffd0747b39 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:07.810827 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-36fea69c-1c3e-48a9-aa0c-67ffd0747b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:07.830897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:07.885545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 32/68] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:07.915740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:07.966518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897ef63-0a87-41e1-bc02-2650455ada0d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:08.017898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e85a0-c682-4eeb-9692-4573498106a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:08.018549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e85a0-c682-4eeb-9692-4573498106a4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.018834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e85a0-c682-4eeb-9692-4573498106a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.019211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e85a0-c682-4eeb-9692-4573498106a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.019878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e85a0-c682-4eeb-9692-4573498106a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:08.020183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e85a0-c682-4eeb-9692-4573498106a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:08.021650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e85a0-c682-4eeb-9692-4573498106a4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:08.032414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 37/69] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:08.038558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fea69c-1c3e-48a9-aa0c-67ffd0747b39 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.072782 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e85a0-c682-4eeb-9692-4573498106a4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.168329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fea69c-1c3e-48a9-aa0c-67ffd0747b39 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:08.206431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e85a0-c682-4eeb-9692-4573498106a4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:08.208867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca6dfe-0f88-4bc4-bfa5-8b103ff05286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:08.209824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca6dfe-0f88-4bc4-bfa5-8b103ff05286 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.209824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca6dfe-0f88-4bc4-bfa5-8b103ff05286 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.210105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca6dfe-0f88-4bc4-bfa5-8b103ff05286 None admin] PATH_INFO: `/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.226408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca6dfe-0f88-4bc4-bfa5-8b103ff05286 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a4ccc7e7914c20a0f1117adfd425c9)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:08.227868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca6dfe-0f88-4bc4-bfa5-8b103ff05286 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:08.233948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 33/70] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:08.233948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9faeb-f515-4733-bd7c-05389088387f None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.233948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9faeb-f515-4733-bd7c-05389088387f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.233948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9faeb-f515-4733-bd7c-05389088387f None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.233948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 34/71] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:08.238677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bc3d2-82fa-48f7-9f3e-add753c92e6e None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.239164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bc3d2-82fa-48f7-9f3e-add753c92e6e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.239386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bc3d2-82fa-48f7-9f3e-add753c92e6e None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.240640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 35/72] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:08.245411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bc3d2-82fa-48f7-9f3e-add753c92e6e None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.248646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d2f3f-6e6e-4b0a-ac07-2edba3199d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:08.249321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d2f3f-6e6e-4b0a-ac07-2edba3199d1a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.249607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d2f3f-6e6e-4b0a-ac07-2edba3199d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.249886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d2f3f-6e6e-4b0a-ac07-2edba3199d1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.252497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:08.260430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:08.261099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d2f3f-6e6e-4b0a-ac07-2edba3199d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-444d2f3f-6e6e-4b0a-ac07-2edba3199d1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:08.266209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:08.267172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:08.268022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 38/73] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:08.268932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a401c-6e76-4944-9c60-ddf2e7d4f4af None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.269247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a401c-6e76-4944-9c60-ddf2e7d4f4af None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.269520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a401c-6e76-4944-9c60-ddf2e7d4f4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.287108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b16a401c-6e76-4944-9c60-ddf2e7d4f4af None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:08.288315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b16a401c-6e76-4944-9c60-ddf2e7d4f4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:08.353028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bc3d2-82fa-48f7-9f3e-add753c92e6e None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:08.354799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 39/74] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:08.356691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a401c-6e76-4944-9c60-ddf2e7d4f4af None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.389438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31d6c-392e-4b55-ac91-69b6f62a2c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:08.390718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31d6c-392e-4b55-ac91-69b6f62a2c73 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.391007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31d6c-392e-4b55-ac91-69b6f62a2c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.391282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31d6c-392e-4b55-ac91-69b6f62a2c73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.463503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b31d6c-392e-4b55-ac91-69b6f62a2c73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:08.463941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a401c-6e76-4944-9c60-ddf2e7d4f4af None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:08.465033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b31d6c-392e-4b55-ac91-69b6f62a2c73 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:08.480041 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:08.490768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:08.499927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bae9f-8032-4dc2-b392-42be3f7398d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:08.500562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bae9f-8032-4dc2-b392-42be3f7398d9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.500869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bae9f-8032-4dc2-b392-42be3f7398d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.501151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bae9f-8032-4dc2-b392-42be3f7398d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.512017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bae9f-8032-4dc2-b392-42be3f7398d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3f9bae9f-8032-4dc2-b392-42be3f7398d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:08.517038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:08.517942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:08.518709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 40/75] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:08.520514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d369f3-a854-4dbb-a9b6-9f7dcc9c084c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.520514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d369f3-a854-4dbb-a9b6-9f7dcc9c084c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.520832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d369f3-a854-4dbb-a9b6-9f7dcc9c084c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.538807 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-16d369f3-a854-4dbb-a9b6-9f7dcc9c084c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:08.540025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-16d369f3-a854-4dbb-a9b6-9f7dcc9c084c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:08.593645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 36/76] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:08.594770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31d6c-392e-4b55-ac91-69b6f62a2c73 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.633785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 41/77] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:08.634438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d369f3-a854-4dbb-a9b6-9f7dcc9c084c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.726015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31d6c-392e-4b55-ac91-69b6f62a2c73 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:08.742911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:08.750443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d369f3-a854-4dbb-a9b6-9f7dcc9c084c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:08.768303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0982fb-99ee-413e-98b2-e8a8eb4e73b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:08.768948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0982fb-99ee-413e-98b2-e8a8eb4e73b2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.769366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0982fb-99ee-413e-98b2-e8a8eb4e73b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.769650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0982fb-99ee-413e-98b2-e8a8eb4e73b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.770416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0982fb-99ee-413e-98b2-e8a8eb4e73b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:08.770633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0982fb-99ee-413e-98b2-e8a8eb4e73b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:08.770900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:08.772044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0982fb-99ee-413e-98b2-e8a8eb4e73b2 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:08.784239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 37/78] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:08.785343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0982fb-99ee-413e-98b2-e8a8eb4e73b2 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.791350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1059db-a77f-45d5-ba23-5d36696cdd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:08.791967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1059db-a77f-45d5-ba23-5d36696cdd63 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.792249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1059db-a77f-45d5-ba23-5d36696cdd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.792520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1059db-a77f-45d5-ba23-5d36696cdd63 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.793225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1059db-a77f-45d5-ba23-5d36696cdd63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:08.793543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1059db-a77f-45d5-ba23-5d36696cdd63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:08.794937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1059db-a77f-45d5-ba23-5d36696cdd63 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:08.805939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 42/79] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:08.806995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1059db-a77f-45d5-ba23-5d36696cdd63 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.907915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0982fb-99ee-413e-98b2-e8a8eb4e73b2 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:08.913966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1059db-a77f-45d5-ba23-5d36696cdd63 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:08.947354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85478c2d-8577-4c63-b1cf-42bd036bd532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:08.947994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85478c2d-8577-4c63-b1cf-42bd036bd532 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.948277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85478c2d-8577-4c63-b1cf-42bd036bd532 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.948554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85478c2d-8577-4c63-b1cf-42bd036bd532 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.948887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89dd68-aafa-46ba-ae33-0d9f6716615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:08.949533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89dd68-aafa-46ba-ae33-0d9f6716615b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:08.949769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85478c2d-8577-4c63-b1cf-42bd036bd532 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:08.949877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89dd68-aafa-46ba-ae33-0d9f6716615b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:08.950150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89dd68-aafa-46ba-ae33-0d9f6716615b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:08.951226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85478c2d-8577-4c63-b1cf-42bd036bd532 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:08.952893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 38/80] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:08.953923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85478c2d-8577-4c63-b1cf-42bd036bd532 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.964519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89dd68-aafa-46ba-ae33-0d9f6716615b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-4f89dd68-aafa-46ba-ae33-0d9f6716615b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:08.969925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:08.970949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:08.972614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 43/81] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:08.977265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89dd68-aafa-46ba-ae33-0d9f6716615b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:08.998675 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:09.017599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:09.056884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85478c2d-8577-4c63-b1cf-42bd036bd532 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.077382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89dd68-aafa-46ba-ae33-0d9f6716615b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.091967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e812d22-64f2-4626-b278-ca16681a3c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.092598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e812d22-64f2-4626-b278-ca16681a3c43 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.092916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e812d22-64f2-4626-b278-ca16681a3c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.093194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e812d22-64f2-4626-b278-ca16681a3c43 None admin] PATH_INFO: `/v3/users/c62c262317454775845cb45d452c8ab2` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.094346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e812d22-64f2-4626-b278-ca16681a3c43 None admin] RBAC: Authorizing `identity:get_user(user_id=c62c262317454775845cb45d452c8ab2)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.095797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e812d22-64f2-4626-b278-ca16681a3c43 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.097519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 39/82] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/users/c62c262317454775845cb45d452c8ab2 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:09.099080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e812d22-64f2-4626-b278-ca16681a3c43 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.116087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c7ae5-72e5-4296-9a2a-99cd106dc8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.116978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c7ae5-72e5-4296-9a2a-99cd106dc8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.117258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c7ae5-72e5-4296-9a2a-99cd106dc8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.117532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c7ae5-72e5-4296-9a2a-99cd106dc8a0 None admin] PATH_INFO: `/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.118656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c7ae5-72e5-4296-9a2a-99cd106dc8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a4ccc7e7914c20a0f1117adfd425c9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.120100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c7ae5-72e5-4296-9a2a-99cd106dc8a0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.121717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 44/83] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:08 2022] GET /identity/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:09.122467 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c7ae5-72e5-4296-9a2a-99cd106dc8a0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.200765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e812d22-64f2-4626-b278-ca16681a3c43 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.211595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:09.221820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c7ae5-72e5-4296-9a2a-99cd106dc8a0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.229939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:09.237360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04113220-d7ce-482c-a367-9b59a79d15fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.238662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113220-d7ce-482c-a367-9b59a79d15fa None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.238956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113220-d7ce-482c-a367-9b59a79d15fa None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.239240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113220-d7ce-482c-a367-9b59a79d15fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.240366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04113220-d7ce-482c-a367-9b59a79d15fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.241854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04113220-d7ce-482c-a367-9b59a79d15fa None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.243433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 40/84] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:09.244437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04113220-d7ce-482c-a367-9b59a79d15fa None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.259602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92adae1-1bd7-44ae-86d3-04134525094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.260211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92adae1-1bd7-44ae-86d3-04134525094d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.260491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92adae1-1bd7-44ae-86d3-04134525094d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.260764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92adae1-1bd7-44ae-86d3-04134525094d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.261514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92adae1-1bd7-44ae-86d3-04134525094d None admin] RBAC: Adding query filter params (role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3, user.id=c7a4ccc7e7914c20a0f1117adfd425c9) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:09.261820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92adae1-1bd7-44ae-86d3-04134525094d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.263268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92adae1-1bd7-44ae-86d3-04134525094d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.268643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 45/85] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:09 2022] GET /identity/v3/role_assignments?user.id=c7a4ccc7e7914c20a0f1117adfd425c9&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:09.270070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92adae1-1bd7-44ae-86d3-04134525094d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.377788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04113220-d7ce-482c-a367-9b59a79d15fa None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.384512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92adae1-1bd7-44ae-86d3-04134525094d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.417146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87224e9-d17b-4a08-8440-daa37a3d3045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.417793 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87224e9-d17b-4a08-8440-daa37a3d3045 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.418076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87224e9-d17b-4a08-8440-daa37a3d3045 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.418354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87224e9-d17b-4a08-8440-daa37a3d3045 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.419039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87224e9-d17b-4a08-8440-daa37a3d3045 None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:09.419348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87224e9-d17b-4a08-8440-daa37a3d3045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.420761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87224e9-d17b-4a08-8440-daa37a3d3045 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.422748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982987e-28ab-4322-8d03-e6dab47ba064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.423408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982987e-28ab-4322-8d03-e6dab47ba064 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.423668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982987e-28ab-4322-8d03-e6dab47ba064 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.423943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982987e-28ab-4322-8d03-e6dab47ba064 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.439832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 41/86] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:09.439832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87224e9-d17b-4a08-8440-daa37a3d3045 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.472655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:09.481662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:09.510758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982987e-28ab-4322-8d03-e6dab47ba064 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.512258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982987e-28ab-4322-8d03-e6dab47ba064 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.565041 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87224e9-d17b-4a08-8440-daa37a3d3045 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.602635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812cf35-91ad-4758-b181-5ca874300832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.603255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812cf35-91ad-4758-b181-5ca874300832 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.603544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812cf35-91ad-4758-b181-5ca874300832 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.603824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812cf35-91ad-4758-b181-5ca874300832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.621404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 46/87] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:09.621404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982987e-28ab-4322-8d03-e6dab47ba064 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.682299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812cf35-91ad-4758-b181-5ca874300832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.683823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812cf35-91ad-4758-b181-5ca874300832 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.689843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:09.723924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982987e-28ab-4322-8d03-e6dab47ba064 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.730482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:09.762201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bd45f-f61f-4fa8-9bce-002da530d1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.762960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd45f-f61f-4fa8-9bce-002da530d1c6 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.763252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd45f-f61f-4fa8-9bce-002da530d1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.763535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd45f-f61f-4fa8-9bce-002da530d1c6 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.764201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bd45f-f61f-4fa8-9bce-002da530d1c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.765690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bd45f-f61f-4fa8-9bce-002da530d1c6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.796441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 42/88] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:09.797104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 47/89] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:42:09 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:42:09.797810 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bd45f-f61f-4fa8-9bce-002da530d1c6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.798317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812cf35-91ad-4758-b181-5ca874300832 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.923860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812cf35-91ad-4758-b181-5ca874300832 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.934832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bd45f-f61f-4fa8-9bce-002da530d1c6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:09.965946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5da40-b87d-4e2e-8eb2-82427c2ff6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.966447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:09.967038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5da40-b87d-4e2e-8eb2-82427c2ff6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.967038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5da40-b87d-4e2e-8eb2-82427c2ff6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.967391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5da40-b87d-4e2e-8eb2-82427c2ff6e6 None admin] PATH_INFO: `/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.968680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5da40-b87d-4e2e-8eb2-82427c2ff6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a4ccc7e7914c20a0f1117adfd425c9)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.970308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5da40-b87d-4e2e-8eb2-82427c2ff6e6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.972272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 43/90] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:09 2022] GET /identity/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:09.973475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5da40-b87d-4e2e-8eb2-82427c2ff6e6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.977284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074dcfe-d269-435d-91f2-d166890cfd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:09.978071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074dcfe-d269-435d-91f2-d166890cfd55 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:09.978724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074dcfe-d269-435d-91f2-d166890cfd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:09.979261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074dcfe-d269-435d-91f2-d166890cfd55 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:09.980773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074dcfe-d269-435d-91f2-d166890cfd55 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:09.982395 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074dcfe-d269-435d-91f2-d166890cfd55 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:09.984150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 48/91] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:09 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:09.985091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074dcfe-d269-435d-91f2-d166890cfd55 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:09.994025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:10.124849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5da40-b87d-4e2e-8eb2-82427c2ff6e6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:10.137909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074dcfe-d269-435d-91f2-d166890cfd55 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:10.174607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ea29c-345b-404e-8801-c50d72898cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:10.175910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ea29c-345b-404e-8801-c50d72898cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:10.176333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ea29c-345b-404e-8801-c50d72898cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:10.176629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ea29c-345b-404e-8801-c50d72898cb1 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:10.177986 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ea29c-345b-404e-8801-c50d72898cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:10.181143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ea29c-345b-404e-8801-c50d72898cb1 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:10.183239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 44/92] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:09 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:10.184448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ea29c-345b-404e-8801-c50d72898cb1 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:10.194966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4447457-6ac1-4f3c-b44b-8655aa4d9741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:10.196268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4447457-6ac1-4f3c-b44b-8655aa4d9741 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:10.196562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4447457-6ac1-4f3c-b44b-8655aa4d9741 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:10.196970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4447457-6ac1-4f3c-b44b-8655aa4d9741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:10.280792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:10.288621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:10.296040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4447457-6ac1-4f3c-b44b-8655aa4d9741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:10.297847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4447457-6ac1-4f3c-b44b-8655aa4d9741 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:10.323368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ea29c-345b-404e-8801-c50d72898cb1 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:10.376197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be234319-b399-4dc8-bb84-9b97203844ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:10.376830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be234319-b399-4dc8-bb84-9b97203844ba None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:10.377158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be234319-b399-4dc8-bb84-9b97203844ba None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:10.378161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be234319-b399-4dc8-bb84-9b97203844ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:10.378161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be234319-b399-4dc8-bb84-9b97203844ba None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3, user.id=c62c262317454775845cb45d452c8ab2) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:10.379053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be234319-b399-4dc8-bb84-9b97203844ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:10.381112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be234319-b399-4dc8-bb84-9b97203844ba None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:10.388189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 45/93] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:10 2022] GET /identity/v3/role_assignments?user.id=c62c262317454775845cb45d452c8ab2&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=0be9777002c641b4a47f78df647cc173 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:10.388189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be234319-b399-4dc8-bb84-9b97203844ba None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:10.426525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 49/94] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:10.433862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4447457-6ac1-4f3c-b44b-8655aa4d9741 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:10.505719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be234319-b399-4dc8-bb84-9b97203844ba None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:10.546409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22d8b4-f64d-4bc5-a3d1-844898db1d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:10.546776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22d8b4-f64d-4bc5-a3d1-844898db1d6d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:10.547074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22d8b4-f64d-4bc5-a3d1-844898db1d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:10.547353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22d8b4-f64d-4bc5-a3d1-844898db1d6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:10.548424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22d8b4-f64d-4bc5-a3d1-844898db1d6d None admin] RBAC: Adding query filter params (role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3, user.id=c7a4ccc7e7914c20a0f1117adfd425c9) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:10.548660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22d8b4-f64d-4bc5-a3d1-844898db1d6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:10.550885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22d8b4-f64d-4bc5-a3d1-844898db1d6d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:10.560831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 46/95] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:10 2022] GET /identity/v3/role_assignments?user.id=c7a4ccc7e7914c20a0f1117adfd425c9&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=3518baf98ba54a28a9c79ec287469aff => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:10.615206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4447457-6ac1-4f3c-b44b-8655aa4d9741 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:10.637243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:10.657853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df752cec-4140-4d8d-939d-9813c53fc3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:10.658986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df752cec-4140-4d8d-939d-9813c53fc3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:10.659485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df752cec-4140-4d8d-939d-9813c53fc3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:10.660535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df752cec-4140-4d8d-939d-9813c53fc3e4 None admin] PATH_INFO: `/v3/projects/7875b9b604ed4c9aa28ac6b5252db272` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:10.662151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df752cec-4140-4d8d-939d-9813c53fc3e4 None admin] RBAC: Authorizing `identity:get_project(project_id=7875b9b604ed4c9aa28ac6b5252db272)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:10.663905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df752cec-4140-4d8d-939d-9813c53fc3e4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:10.666415 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 50/96] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:10 2022] GET /identity/v3/projects/7875b9b604ed4c9aa28ac6b5252db272 => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:10.672948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22d8b4-f64d-4bc5-a3d1-844898db1d6d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:10.681685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:10.823064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22d8b4-f64d-4bc5-a3d1-844898db1d6d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:10.877498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc92a5-88c6-4b2d-aab3-7169ff15feda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:10.879165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc92a5-88c6-4b2d-aab3-7169ff15feda None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:10.879653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc92a5-88c6-4b2d-aab3-7169ff15feda None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:10.880130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc92a5-88c6-4b2d-aab3-7169ff15feda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:10.881169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc92a5-88c6-4b2d-aab3-7169ff15feda None admin] RBAC: Adding query filter params (role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=7875b9b604ed4c9aa28ac6b5252db272, user.id=c7a4ccc7e7914c20a0f1117adfd425c9) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:10.881689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc92a5-88c6-4b2d-aab3-7169ff15feda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:10.883394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc92a5-88c6-4b2d-aab3-7169ff15feda None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:10.889556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 47/97] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:10 2022] GET /identity/v3/role_assignments?user.id=c7a4ccc7e7914c20a0f1117adfd425c9&scope.project.id=7875b9b604ed4c9aa28ac6b5252db272&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:11.922029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd67e8-386b-48ef-82cb-1613448eb913 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:11.922573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd67e8-386b-48ef-82cb-1613448eb913 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:11.922573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd67e8-386b-48ef-82cb-1613448eb913 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:11.923948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 51/98] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:11.930026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adad06-a29a-48e7-9086-059a1998ad6b None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:11.930416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adad06-a29a-48e7-9086-059a1998ad6b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:11.930597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adad06-a29a-48e7-9086-059a1998ad6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:11.954128 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-f9adad06-a29a-48e7-9086-059a1998ad6b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:11.955402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f9adad06-a29a-48e7-9086-059a1998ad6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:12.042089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:12.048121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 48/99] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:12.616306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd67e8-386b-48ef-82cb-1613448eb913 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:12.734763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd67e8-386b-48ef-82cb-1613448eb913 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:12.781628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638b2f-53d9-4894-b928-43a338972a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:12.782277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638b2f-53d9-4894-b928-43a338972a10 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:12.782557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638b2f-53d9-4894-b928-43a338972a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:12.782832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638b2f-53d9-4894-b928-43a338972a10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:12.797800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48638b2f-53d9-4894-b928-43a338972a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-48638b2f-53d9-4894-b928-43a338972a10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:12.803428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:12.804319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:12.805535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 52/100] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:12.810662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adad06-a29a-48e7-9086-059a1998ad6b None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:12.917369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adad06-a29a-48e7-9086-059a1998ad6b None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:12.960475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9759e5d-db7f-43d9-afe2-3e08db7d6487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:12.961262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9759e5d-db7f-43d9-afe2-3e08db7d6487 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:12.961419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9759e5d-db7f-43d9-afe2-3e08db7d6487 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:12.961663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9759e5d-db7f-43d9-afe2-3e08db7d6487 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:12.962335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9759e5d-db7f-43d9-afe2-3e08db7d6487 None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:12.962639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9759e5d-db7f-43d9-afe2-3e08db7d6487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:12.964042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9759e5d-db7f-43d9-afe2-3e08db7d6487 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:12.972943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 49/101] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:12.978544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638b2f-53d9-4894-b928-43a338972a10 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:13.004962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:13.095855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638b2f-53d9-4894-b928-43a338972a10 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:13.136427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b09121-6bad-4341-9491-7ec323e29235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:13.137168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09121-6bad-4341-9491-7ec323e29235 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:13.137495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09121-6bad-4341-9491-7ec323e29235 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:13.137777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09121-6bad-4341-9491-7ec323e29235 None admin] PATH_INFO: `/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:13.138915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b09121-6bad-4341-9491-7ec323e29235 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a4ccc7e7914c20a0f1117adfd425c9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:13.140383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b09121-6bad-4341-9491-7ec323e29235 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:13.142045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 53/102] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:12 2022] GET /identity/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:13.147811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9759e5d-db7f-43d9-afe2-3e08db7d6487 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:13.239828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:13.298923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9759e5d-db7f-43d9-afe2-3e08db7d6487 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:13.350648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f46679-1d4b-4c38-aec1-2119be6ecbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:13.350819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271c12f-90eb-4544-86cd-9c4ea4dc907a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:13.351051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271c12f-90eb-4544-86cd-9c4ea4dc907a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:13.351202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271c12f-90eb-4544-86cd-9c4ea4dc907a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:13.352054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f46679-1d4b-4c38-aec1-2119be6ecbcc None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:13.352530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f46679-1d4b-4c38-aec1-2119be6ecbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:13.352880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 54/103] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:13.353276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f46679-1d4b-4c38-aec1-2119be6ecbcc None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:13.356687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f46679-1d4b-4c38-aec1-2119be6ecbcc None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:13.358467 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a46a4-496d-4bdb-be56-2fd8470f5360 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:13.358759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a46a4-496d-4bdb-be56-2fd8470f5360 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:13.359041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a46a4-496d-4bdb-be56-2fd8470f5360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:13.359523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f46679-1d4b-4c38-aec1-2119be6ecbcc None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:13.361509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 50/104] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:13 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:13.367917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f46679-1d4b-4c38-aec1-2119be6ecbcc None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:13.386575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-637a46a4-496d-4bdb-be56-2fd8470f5360 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:13.388088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-637a46a4-496d-4bdb-be56-2fd8470f5360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:13.520221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 55/105] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:13.525687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fb0c5-1bdc-4c7b-ad83-a3dc0bc76030 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:13.526095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fb0c5-1bdc-4c7b-ad83-a3dc0bc76030 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:13.526449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fb0c5-1bdc-4c7b-ad83-a3dc0bc76030 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:13.527883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 56/106] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:13.529198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98dba4-0767-475a-82b3-0886a557c290 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:13.529512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98dba4-0767-475a-82b3-0886a557c290 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:13.529862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98dba4-0767-475a-82b3-0886a557c290 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:13.531215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 57/107] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:13.541832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d5053-c3bb-4705-9fe2-e04b8a18c432 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:13.542193 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d5053-c3bb-4705-9fe2-e04b8a18c432 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:13.542544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d5053-c3bb-4705-9fe2-e04b8a18c432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:13.543168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f46679-1d4b-4c38-aec1-2119be6ecbcc None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:13.572959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:13.574219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-416d5053-c3bb-4705-9fe2-e04b8a18c432 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:13.576027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-416d5053-c3bb-4705-9fe2-e04b8a18c432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:13.582524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e8316-c00b-45b8-92bf-c77bbd104587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:13.583298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e8316-c00b-45b8-92bf-c77bbd104587 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:13.583584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e8316-c00b-45b8-92bf-c77bbd104587 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:13.583864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e8316-c00b-45b8-92bf-c77bbd104587 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/users/c7a4ccc7e7914c20a0f1117adfd425c9/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:13.585866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e8316-c00b-45b8-92bf-c77bbd104587 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9cd5f1606ad462482bd725a3a3537a3, user_id=c7a4ccc7e7914c20a0f1117adfd425c9, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:13.587446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e8316-c00b-45b8-92bf-c77bbd104587 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:13.595586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:13.606208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 51/108] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:42:13 2022] PUT /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/users/c7a4ccc7e7914c20a0f1117adfd425c9/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:42:13.607763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516238c-a430-4065-b57e-f1a6047635c6 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:13.608039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516238c-a430-4065-b57e-f1a6047635c6 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:13.608322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516238c-a430-4065-b57e-f1a6047635c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:13.630149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-8516238c-a430-4065-b57e-f1a6047635c6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:13.631618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8516238c-a430-4065-b57e-f1a6047635c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:13.663117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 58/109] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:13.714709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 52/110] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:13.967266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d5053-c3bb-4705-9fe2-e04b8a18c432 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:14.081722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516238c-a430-4065-b57e-f1a6047635c6 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:14.094684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d5053-c3bb-4705-9fe2-e04b8a18c432 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:14.169126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3b17a-5083-496c-a1bc-c19751e96637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:14.169792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3b17a-5083-496c-a1bc-c19751e96637 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.170081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3b17a-5083-496c-a1bc-c19751e96637 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.170361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3b17a-5083-496c-a1bc-c19751e96637 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.197793 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3b17a-5083-496c-a1bc-c19751e96637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-a1b3b17a-5083-496c-a1bc-c19751e96637 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:14.202852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:14.203998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:14.204764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 59/111] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:42:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:14.206366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a2461-3956-491a-82be-7917c124640a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.206655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a2461-3956-491a-82be-7917c124640a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.206936 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a2461-3956-491a-82be-7917c124640a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.208478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 60/112] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:14.214001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364ee49-f567-4050-8342-43ce82c803d5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.214293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364ee49-f567-4050-8342-43ce82c803d5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.214575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364ee49-f567-4050-8342-43ce82c803d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.246411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6364ee49-f567-4050-8342-43ce82c803d5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:14.250477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6364ee49-f567-4050-8342-43ce82c803d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:14.255302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:14.287121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516238c-a430-4065-b57e-f1a6047635c6 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:14.313505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:14.338181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62bd3d-e399-42fc-be93-80faf3ee6ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:14.339183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62bd3d-e399-42fc-be93-80faf3ee6ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.339661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62bd3d-e399-42fc-be93-80faf3ee6ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.340154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62bd3d-e399-42fc-be93-80faf3ee6ed3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.341532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62bd3d-e399-42fc-be93-80faf3ee6ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:14.343729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62bd3d-e399-42fc-be93-80faf3ee6ed3 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:14.345835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 53/113] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:14.346799 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62bd3d-e399-42fc-be93-80faf3ee6ed3 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:14.355335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 61/114] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:14.356185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364ee49-f567-4050-8342-43ce82c803d5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:14.487723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62bd3d-e399-42fc-be93-80faf3ee6ed3 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:14.510152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364ee49-f567-4050-8342-43ce82c803d5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:14.529320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139be179-0a7d-465d-964a-07299cf3adf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:14.530001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139be179-0a7d-465d-964a-07299cf3adf6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.530284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139be179-0a7d-465d-964a-07299cf3adf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.530551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139be179-0a7d-465d-964a-07299cf3adf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.531228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139be179-0a7d-465d-964a-07299cf3adf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:14.531536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139be179-0a7d-465d-964a-07299cf3adf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:14.532966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139be179-0a7d-465d-964a-07299cf3adf6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:14.547609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 54/115] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:14.548422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139be179-0a7d-465d-964a-07299cf3adf6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:14.566293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b5b5c-264b-42f6-80f6-4c3e1ce05f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:14.567116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b5b5c-264b-42f6-80f6-4c3e1ce05f3a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.567428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b5b5c-264b-42f6-80f6-4c3e1ce05f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.567708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b5b5c-264b-42f6-80f6-4c3e1ce05f3a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.575883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:14.579294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b5b5c-264b-42f6-80f6-4c3e1ce05f3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-809b5b5c-264b-42f6-80f6-4c3e1ce05f3a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:14.584899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:14.585963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:14.586927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:14.593613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 62/116] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:14.594656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106594b4-ac07-474a-8569-f522d70afc29 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.595034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106594b4-ac07-474a-8569-f522d70afc29 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.595416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106594b4-ac07-474a-8569-f522d70afc29 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.597512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 63/117] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:14.598375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106594b4-ac07-474a-8569-f522d70afc29 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:14.618513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:14.679664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139be179-0a7d-465d-964a-07299cf3adf6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:14.710046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106594b4-ac07-474a-8569-f522d70afc29 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:14.717697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0c3c8-cb43-4263-9e13-831af7b98688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:14.718324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0c3c8-cb43-4263-9e13-831af7b98688 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.718612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0c3c8-cb43-4263-9e13-831af7b98688 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.718894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0c3c8-cb43-4263-9e13-831af7b98688 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.719560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0c3c8-cb43-4263-9e13-831af7b98688 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:14.721063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0c3c8-cb43-4263-9e13-831af7b98688 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:14.749340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645aecd-0460-4f16-9ca3-71f5cdce15b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:14.757030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645aecd-0460-4f16-9ca3-71f5cdce15b6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.757220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 55/118] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:42:14 2022] POST /identity/v3/projects => generated 302 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:42:14.757312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645aecd-0460-4f16-9ca3-71f5cdce15b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.757312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645aecd-0460-4f16-9ca3-71f5cdce15b6 None admin] PATH_INFO: `/v3/users/c62c262317454775845cb45d452c8ab2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.757481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0c3c8-cb43-4263-9e13-831af7b98688 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:14.758582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645aecd-0460-4f16-9ca3-71f5cdce15b6 None admin] RBAC: Authorizing `identity:get_user(user_id=c62c262317454775845cb45d452c8ab2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:14.760364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645aecd-0460-4f16-9ca3-71f5cdce15b6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:14.763017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 64/119] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:14 2022] GET /identity/v3/users/c62c262317454775845cb45d452c8ab2 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:14.763335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936ad86-de25-44aa-b289-78c5e89ab37a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.763626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936ad86-de25-44aa-b289-78c5e89ab37a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.763903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936ad86-de25-44aa-b289-78c5e89ab37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.785147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4936ad86-de25-44aa-b289-78c5e89ab37a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:14.786388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4936ad86-de25-44aa-b289-78c5e89ab37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:14.869079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:14.875106 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 65/120] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:14.876250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936ad86-de25-44aa-b289-78c5e89ab37a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:14.883954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:14.887697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0c3c8-cb43-4263-9e13-831af7b98688 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:14.934058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ab9bf-1b85-4a55-90f0-fad507360a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:14.934710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ab9bf-1b85-4a55-90f0-fad507360a6f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:14.934997 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ab9bf-1b85-4a55-90f0-fad507360a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:14.935277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ab9bf-1b85-4a55-90f0-fad507360a6f None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:14.935972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ab9bf-1b85-4a55-90f0-fad507360a6f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:14.936529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ab9bf-1b85-4a55-90f0-fad507360a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:14.938264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ab9bf-1b85-4a55-90f0-fad507360a6f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:14.950076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 56/121] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:14.951114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ab9bf-1b85-4a55-90f0-fad507360a6f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:15.046381 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936ad86-de25-44aa-b289-78c5e89ab37a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:15.076843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ab9bf-1b85-4a55-90f0-fad507360a6f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:15.084947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f141d1-cca3-4e29-9da2-3a45a78bfe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:15.085930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f141d1-cca3-4e29-9da2-3a45a78bfe93 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:15.086412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f141d1-cca3-4e29-9da2-3a45a78bfe93 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:15.086885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f141d1-cca3-4e29-9da2-3a45a78bfe93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:15.098956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f141d1-cca3-4e29-9da2-3a45a78bfe93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e4f141d1-cca3-4e29-9da2-3a45a78bfe93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:15.105970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:15.106815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:15.107606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:15.110073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 66/122] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:15.111156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f141d1-cca3-4e29-9da2-3a45a78bfe93 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:15.115156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9568c0-b021-4a92-9808-6a039183e467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:15.115884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9568c0-b021-4a92-9808-6a039183e467 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:15.116173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9568c0-b021-4a92-9808-6a039183e467 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:15.116451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9568c0-b021-4a92-9808-6a039183e467 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:15.117783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9568c0-b021-4a92-9808-6a039183e467 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:15.119395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9568c0-b021-4a92-9808-6a039183e467 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:15.121224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 57/123] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:14 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:15.121863 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9568c0-b021-4a92-9808-6a039183e467 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:15.157075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:15.188649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:15.245247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f141d1-cca3-4e29-9da2-3a45a78bfe93 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:15.246419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9568c0-b021-4a92-9808-6a039183e467 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:15.284164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d32c52-9b0a-4cf4-889b-c2f08aa1f856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:15.285140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe8cc3-874e-4613-a885-d6f5f0588180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:15.285589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d32c52-9b0a-4cf4-889b-c2f08aa1f856 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:15.285755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe8cc3-874e-4613-a885-d6f5f0588180 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:15.286041 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe8cc3-874e-4613-a885-d6f5f0588180 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:15.286316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe8cc3-874e-4613-a885-d6f5f0588180 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:15.286459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d32c52-9b0a-4cf4-889b-c2f08aa1f856 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:15.286896 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d32c52-9b0a-4cf4-889b-c2f08aa1f856 None admin] PATH_INFO: `/v3/users/1e91f8a4248f4ad7886e2a17ed75a4d4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:15.286999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe8cc3-874e-4613-a885-d6f5f0588180 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:15.287275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe8cc3-874e-4613-a885-d6f5f0588180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:15.288847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe8cc3-874e-4613-a885-d6f5f0588180 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:15.297325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 58/124] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:15.298059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe8cc3-874e-4613-a885-d6f5f0588180 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:15.304378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d32c52-9b0a-4cf4-889b-c2f08aa1f856 None admin] RBAC: Authorizing `identity:get_user(user_id=1e91f8a4248f4ad7886e2a17ed75a4d4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:15.306078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d32c52-9b0a-4cf4-889b-c2f08aa1f856 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:15.308234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 67/125] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:15 2022] GET /identity/v3/users/1e91f8a4248f4ad7886e2a17ed75a4d4 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:15.313649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d32c52-9b0a-4cf4-889b-c2f08aa1f856 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:15.400667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:15.411094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe8cc3-874e-4613-a885-d6f5f0588180 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:15.434713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d32c52-9b0a-4cf4-889b-c2f08aa1f856 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:15.451191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3ab3d-fe7a-4269-a681-8c2443560f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:15.451868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3ab3d-fe7a-4269-a681-8c2443560f45 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:15.452153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3ab3d-fe7a-4269-a681-8c2443560f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:15.452559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3ab3d-fe7a-4269-a681-8c2443560f45 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/users/c62c262317454775845cb45d452c8ab2/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:15.454506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:15.454783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3ab3d-fe7a-4269-a681-8c2443560f45 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9cd5f1606ad462482bd725a3a3537a3, user_id=c62c262317454775845cb45d452c8ab2, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:15.456941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3ab3d-fe7a-4269-a681-8c2443560f45 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:15.480006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 59/126] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:42:15 2022] PUT /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/users/c62c262317454775845cb45d452c8ab2/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:15.485070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea8c2c-b8ad-43a5-a25e-e744a7250905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:15.485070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea8c2c-b8ad-43a5-a25e-e744a7250905 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:15.485070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea8c2c-b8ad-43a5-a25e-e744a7250905 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:15.485070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea8c2c-b8ad-43a5-a25e-e744a7250905 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:15.486330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3ab3d-fe7a-4269-a681-8c2443560f45 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:15.509637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea8c2c-b8ad-43a5-a25e-e744a7250905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ebea8c2c-b8ad-43a5-a25e-e744a7250905 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:15.514986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:15.516023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:15.516867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 68/127] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:15.518076 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea8c2c-b8ad-43a5-a25e-e744a7250905 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:15.707482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea8c2c-b8ad-43a5-a25e-e744a7250905 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:15.713131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3ab3d-fe7a-4269-a681-8c2443560f45 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:15.748161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58061a5e-732e-4f6b-a558-07eef2724dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:15.748956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58061a5e-732e-4f6b-a558-07eef2724dfd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:15.749456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58061a5e-732e-4f6b-a558-07eef2724dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:15.749828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58061a5e-732e-4f6b-a558-07eef2724dfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:15.756651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e62b1-128e-444c-99d1-7ea35220a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:15.757519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e62b1-128e-444c-99d1-7ea35220a204 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:15.757967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e62b1-128e-444c-99d1-7ea35220a204 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:15.758383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e62b1-128e-444c-99d1-7ea35220a204 None admin] PATH_INFO: `/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:15.759690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e62b1-128e-444c-99d1-7ea35220a204 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a4ccc7e7914c20a0f1117adfd425c9)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:15.761995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e62b1-128e-444c-99d1-7ea35220a204 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:15.764531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 60/128] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:15 2022] GET /identity/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9 => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:15.765561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e62b1-128e-444c-99d1-7ea35220a204 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:15.794220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:15.811586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:15.855828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58061a5e-732e-4f6b-a558-07eef2724dfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:15.857365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58061a5e-732e-4f6b-a558-07eef2724dfd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:15.958298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e62b1-128e-444c-99d1-7ea35220a204 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:15.985137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 69/129] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:15.986311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58061a5e-732e-4f6b-a558-07eef2724dfd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:16.058136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f74736-8db5-402c-9b2e-5f8995c57eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:16.061382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f74736-8db5-402c-9b2e-5f8995c57eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:16.061939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f74736-8db5-402c-9b2e-5f8995c57eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:16.062439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f74736-8db5-402c-9b2e-5f8995c57eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:16.063324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f74736-8db5-402c-9b2e-5f8995c57eb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:16.064399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f74736-8db5-402c-9b2e-5f8995c57eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:16.066542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f74736-8db5-402c-9b2e-5f8995c57eb7 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:16.077704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 61/130] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:16.079194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f74736-8db5-402c-9b2e-5f8995c57eb7 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:16.095710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:16.175637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58061a5e-732e-4f6b-a558-07eef2724dfd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:16.179679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:16.214356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f74736-8db5-402c-9b2e-5f8995c57eb7 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:16.247961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbca7b-8056-4f84-bda9-31c2b7575c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:16.248614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbca7b-8056-4f84-bda9-31c2b7575c89 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:16.248928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbca7b-8056-4f84-bda9-31c2b7575c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:16.249207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbca7b-8056-4f84-bda9-31c2b7575c89 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:16.251117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbca7b-8056-4f84-bda9-31c2b7575c89 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:16.252603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbca7b-8056-4f84-bda9-31c2b7575c89 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:16.258566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 70/131] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:15 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:16.260418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbca7b-8056-4f84-bda9-31c2b7575c89 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:16.291565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040adfc5-6266-4e9e-90a6-1c2d4f8037b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:16.292236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040adfc5-6266-4e9e-90a6-1c2d4f8037b0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:16.292521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040adfc5-6266-4e9e-90a6-1c2d4f8037b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:16.292816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040adfc5-6266-4e9e-90a6-1c2d4f8037b0 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:16.297599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040adfc5-6266-4e9e-90a6-1c2d4f8037b0 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:16.299093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040adfc5-6266-4e9e-90a6-1c2d4f8037b0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:16.301666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 62/132] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:16 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:16.302509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040adfc5-6266-4e9e-90a6-1c2d4f8037b0 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:16.386794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbca7b-8056-4f84-bda9-31c2b7575c89 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:16.434645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a553149-7379-45c0-9dde-589306ed1085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:16.435563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a553149-7379-45c0-9dde-589306ed1085 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:16.435848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a553149-7379-45c0-9dde-589306ed1085 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:16.436124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a553149-7379-45c0-9dde-589306ed1085 None admin] PATH_INFO: `/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:16.437548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a553149-7379-45c0-9dde-589306ed1085 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a4ccc7e7914c20a0f1117adfd425c9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:16.439733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a553149-7379-45c0-9dde-589306ed1085 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:16.442481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 71/133] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:16 2022] GET /identity/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:16.443102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a553149-7379-45c0-9dde-589306ed1085 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:16.462674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:16.466174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040adfc5-6266-4e9e-90a6-1c2d4f8037b0 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:16.508664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed1af4-0692-4e67-a630-594cd6e3f344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:16.509856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed1af4-0692-4e67-a630-594cd6e3f344 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:16.510371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed1af4-0692-4e67-a630-594cd6e3f344 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:16.510884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed1af4-0692-4e67-a630-594cd6e3f344 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/users/c7a4ccc7e7914c20a0f1117adfd425c9/roles/3518baf98ba54a28a9c79ec287469aff` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:16.513792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed1af4-0692-4e67-a630-594cd6e3f344 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9cd5f1606ad462482bd725a3a3537a3, user_id=c7a4ccc7e7914c20a0f1117adfd425c9, role_id=3518baf98ba54a28a9c79ec287469aff)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:16.515496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed1af4-0692-4e67-a630-594cd6e3f344 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:16.535799 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 63/134] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:42:16 2022] PUT /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/users/c7a4ccc7e7914c20a0f1117adfd425c9/roles/3518baf98ba54a28a9c79ec287469aff => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:16.537469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39899c5-c56f-429c-a682-9c8f38219b8a None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:16.537981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39899c5-c56f-429c-a682-9c8f38219b8a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:16.538521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39899c5-c56f-429c-a682-9c8f38219b8a None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:16.540139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 64/135] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:16.541259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39899c5-c56f-429c-a682-9c8f38219b8a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:16.562417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:16.589489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a553149-7379-45c0-9dde-589306ed1085 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:16.663743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f325b-c8a5-45df-b08b-34bf4725e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:16.664876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f325b-c8a5-45df-b08b-34bf4725e785 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:16.665462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f325b-c8a5-45df-b08b-34bf4725e785 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:16.666698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f325b-c8a5-45df-b08b-34bf4725e785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:16.667687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f325b-c8a5-45df-b08b-34bf4725e785 None admin] RBAC: Adding query filter params (role.id=fdf0d588fb134183911419bbaa627d22, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3, user.id=1e91f8a4248f4ad7886e2a17ed75a4d4) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:16.669388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f325b-c8a5-45df-b08b-34bf4725e785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:16.671098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f325b-c8a5-45df-b08b-34bf4725e785 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:16.687209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 72/136] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:16 2022] GET /identity/v3/role_assignments?user.id=1e91f8a4248f4ad7886e2a17ed75a4d4&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=fdf0d588fb134183911419bbaa627d22 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:16.689063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0d01e-a0a0-41d9-9f48-8d0a75036d31 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:16.689595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0d01e-a0a0-41d9-9f48-8d0a75036d31 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:16.692849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0d01e-a0a0-41d9-9f48-8d0a75036d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:16.730506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dda0d01e-a0a0-41d9-9f48-8d0a75036d31 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:16.735376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dda0d01e-a0a0-41d9-9f48-8d0a75036d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:16.764362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39899c5-c56f-429c-a682-9c8f38219b8a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:16.839576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e589e-9b95-4fd1-9eb0-b9fa39ba8967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:16.840654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e589e-9b95-4fd1-9eb0-b9fa39ba8967 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:16.841166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e589e-9b95-4fd1-9eb0-b9fa39ba8967 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:16.841675 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e589e-9b95-4fd1-9eb0-b9fa39ba8967 None admin] PATH_INFO: `/v3/projects/7875b9b604ed4c9aa28ac6b5252db272` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:16.843222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e589e-9b95-4fd1-9eb0-b9fa39ba8967 None admin] RBAC: Authorizing `identity:get_project(project_id=7875b9b604ed4c9aa28ac6b5252db272)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:16.847548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e589e-9b95-4fd1-9eb0-b9fa39ba8967 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:16.853668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 65/137] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:16 2022] GET /identity/v3/projects/7875b9b604ed4c9aa28ac6b5252db272 => generated 312 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:16.860206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e589e-9b95-4fd1-9eb0-b9fa39ba8967 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:16.891254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 73/138] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:16.965144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:16.993495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e589e-9b95-4fd1-9eb0-b9fa39ba8967 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:17.077660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc5b32-36f9-4ea5-8c6e-1222b5b98a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:17.079052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc5b32-36f9-4ea5-8c6e-1222b5b98a8d None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:17.079546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc5b32-36f9-4ea5-8c6e-1222b5b98a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:17.080028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc5b32-36f9-4ea5-8c6e-1222b5b98a8d None admin] PATH_INFO: `/v3/projects/7875b9b604ed4c9aa28ac6b5252db272/users/c7a4ccc7e7914c20a0f1117adfd425c9/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:17.089550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc5b32-36f9-4ea5-8c6e-1222b5b98a8d None admin] RBAC: Authorizing `identity:create_grant(project_id=7875b9b604ed4c9aa28ac6b5252db272, user_id=c7a4ccc7e7914c20a0f1117adfd425c9, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:17.091317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc5b32-36f9-4ea5-8c6e-1222b5b98a8d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:17.129621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 66/139] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:42:16 2022] PUT /identity/v3/projects/7875b9b604ed4c9aa28ac6b5252db272/users/c7a4ccc7e7914c20a0f1117adfd425c9/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:42:17.428312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0d01e-a0a0-41d9-9f48-8d0a75036d31 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:17.437661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:17.604351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0d01e-a0a0-41d9-9f48-8d0a75036d31 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:17.671393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a3dc6-3fda-40b4-b9b6-659240bb81ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:17.672105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a3dc6-3fda-40b4-b9b6-659240bb81ae None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:17.672391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a3dc6-3fda-40b4-b9b6-659240bb81ae None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:17.672667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a3dc6-3fda-40b4-b9b6-659240bb81ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:17.700300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a3dc6-3fda-40b4-b9b6-659240bb81ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3e7a3dc6-3fda-40b4-b9b6-659240bb81ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:17.709383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:17.710309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:17.711514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 74/140] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:17.717364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc5b32-36f9-4ea5-8c6e-1222b5b98a8d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:17.906752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc5b32-36f9-4ea5-8c6e-1222b5b98a8d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:17.953823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecee4a0-98b9-46f0-8a35-4cf10bb1488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:17.954343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecee4a0-98b9-46f0-8a35-4cf10bb1488f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:17.954690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecee4a0-98b9-46f0-8a35-4cf10bb1488f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:17.954971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecee4a0-98b9-46f0-8a35-4cf10bb1488f None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:17.955653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecee4a0-98b9-46f0-8a35-4cf10bb1488f None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:17.955959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecee4a0-98b9-46f0-8a35-4cf10bb1488f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:17.957971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecee4a0-98b9-46f0-8a35-4cf10bb1488f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:17.965750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:17.970589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 67/141] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:17.977167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a3dc6-3fda-40b4-b9b6-659240bb81ae None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:18.007308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7cfb8-4d06-4190-af0a-2feedb84b637 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:18.007611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7cfb8-4d06-4190-af0a-2feedb84b637 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:18.007863 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7cfb8-4d06-4190-af0a-2feedb84b637 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:18.009279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 68/142] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:18.016463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937322c4-694f-408a-8bf5-1d41ee44f4ed None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:18.017054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937322c4-694f-408a-8bf5-1d41ee44f4ed None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:18.017495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937322c4-694f-408a-8bf5-1d41ee44f4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:18.063063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-937322c4-694f-408a-8bf5-1d41ee44f4ed None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:18.064276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-937322c4-694f-408a-8bf5-1d41ee44f4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:18.117171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:18.215893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a3dc6-3fda-40b4-b9b6-659240bb81ae None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:18.246070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 69/143] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:18.288477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4456ac-c7d7-45b9-8caa-7d190002ed72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:18.289153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4456ac-c7d7-45b9-8caa-7d190002ed72 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:18.289494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4456ac-c7d7-45b9-8caa-7d190002ed72 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:18.289772 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4456ac-c7d7-45b9-8caa-7d190002ed72 None admin] PATH_INFO: `/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:18.293524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4456ac-c7d7-45b9-8caa-7d190002ed72 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a4ccc7e7914c20a0f1117adfd425c9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:18.295013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4456ac-c7d7-45b9-8caa-7d190002ed72 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:18.296602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 75/144] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:17 2022] GET /identity/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9 => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:18.302842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937322c4-694f-408a-8bf5-1d41ee44f4ed None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:18.314026 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2bc78-0472-4c27-b10c-9da445683b9e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:18.314458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2bc78-0472-4c27-b10c-9da445683b9e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:18.314731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2bc78-0472-4c27-b10c-9da445683b9e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:18.316979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 76/145] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:42:18.322102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883d696-0826-49f0-91a3-4961067cda72 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:18.322384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883d696-0826-49f0-91a3-4961067cda72 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:18.322655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883d696-0826-49f0-91a3-4961067cda72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:18.341523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3883d696-0826-49f0-91a3-4961067cda72 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:18.342734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3883d696-0826-49f0-91a3-4961067cda72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:18.415582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 77/146] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:18.498187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937322c4-694f-408a-8bf5-1d41ee44f4ed None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:18.532322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:18.542481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10ed43-a1c0-43aa-8a66-317f6b6fb6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:18.543122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10ed43-a1c0-43aa-8a66-317f6b6fb6ef None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:18.543408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10ed43-a1c0-43aa-8a66-317f6b6fb6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:18.543688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10ed43-a1c0-43aa-8a66-317f6b6fb6ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:18.638314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10ed43-a1c0-43aa-8a66-317f6b6fb6ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:18.639805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10ed43-a1c0-43aa-8a66-317f6b6fb6ef None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:18.741860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 70/147] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:18.747580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883d696-0826-49f0-91a3-4961067cda72 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:18.769743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10ed43-a1c0-43aa-8a66-317f6b6fb6ef None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:18.802149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:18.853835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:18.878895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10ed43-a1c0-43aa-8a66-317f6b6fb6ef None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:18.915314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e521b2c-1c1d-4508-8127-46cc048386dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:18.915956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e521b2c-1c1d-4508-8127-46cc048386dc None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:18.916245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e521b2c-1c1d-4508-8127-46cc048386dc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:18.916521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e521b2c-1c1d-4508-8127-46cc048386dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:18.930316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e521b2c-1c1d-4508-8127-46cc048386dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-6e521b2c-1c1d-4508-8127-46cc048386dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:18.935268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:18.936140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:18.936942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:18.938434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 71/148] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:42:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:18.939037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e521b2c-1c1d-4508-8127-46cc048386dc None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:18.944877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883d696-0826-49f0-91a3-4961067cda72 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:19.005697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c884f6f-6ce4-4b3b-94f3-40ebed63234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:19.006469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c884f6f-6ce4-4b3b-94f3-40ebed63234f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:19.006749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c884f6f-6ce4-4b3b-94f3-40ebed63234f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:19.007270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c884f6f-6ce4-4b3b-94f3-40ebed63234f None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:19.008398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c884f6f-6ce4-4b3b-94f3-40ebed63234f None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:19.009898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c884f6f-6ce4-4b3b-94f3-40ebed63234f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:19.011753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 78/149] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:18 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:19.012813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c884f6f-6ce4-4b3b-94f3-40ebed63234f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:19.143071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:19.149395 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c884f6f-6ce4-4b3b-94f3-40ebed63234f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:19.165619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e521b2c-1c1d-4508-8127-46cc048386dc None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:19.181468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:19.188038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69590fa8-1368-465d-ab28-b201faa605b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:19.188655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69590fa8-1368-465d-ab28-b201faa605b6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:19.188963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69590fa8-1368-465d-ab28-b201faa605b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:19.189238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69590fa8-1368-465d-ab28-b201faa605b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:19.189908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69590fa8-1368-465d-ab28-b201faa605b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:19.190215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69590fa8-1368-465d-ab28-b201faa605b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:19.191873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69590fa8-1368-465d-ab28-b201faa605b6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:19.200832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 79/150] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:19.201661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69590fa8-1368-465d-ab28-b201faa605b6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:19.262940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab69c64-0f89-4711-b651-90fde102a063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:19.263587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab69c64-0f89-4711-b651-90fde102a063 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:19.263871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab69c64-0f89-4711-b651-90fde102a063 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:19.264144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab69c64-0f89-4711-b651-90fde102a063 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:19.269787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab69c64-0f89-4711-b651-90fde102a063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:19.271202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab69c64-0f89-4711-b651-90fde102a063 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:19.273136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 72/151] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:19.273925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab69c64-0f89-4711-b651-90fde102a063 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:19.312787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69590fa8-1368-465d-ab28-b201faa605b6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:19.354293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dc881-dc0a-4972-9219-1246fd1dc5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:19.354956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dc881-dc0a-4972-9219-1246fd1dc5bf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:19.355238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dc881-dc0a-4972-9219-1246fd1dc5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:19.355561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dc881-dc0a-4972-9219-1246fd1dc5bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:19.356294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59dc881-dc0a-4972-9219-1246fd1dc5bf None admin] RBAC: Adding query filter params (role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3, user.id=c7a4ccc7e7914c20a0f1117adfd425c9) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:19.356604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59dc881-dc0a-4972-9219-1246fd1dc5bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:19.358422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59dc881-dc0a-4972-9219-1246fd1dc5bf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:19.366256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 80/152] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:19 2022] GET /identity/v3/role_assignments?user.id=c7a4ccc7e7914c20a0f1117adfd425c9&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:19.366873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dc881-dc0a-4972-9219-1246fd1dc5bf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:19.517768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab69c64-0f89-4711-b651-90fde102a063 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:19.519826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:19.615427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e83ab-4282-4065-a11b-9fde29b3c390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:19.616426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e83ab-4282-4065-a11b-9fde29b3c390 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:19.616950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e83ab-4282-4065-a11b-9fde29b3c390 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:19.617417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e83ab-4282-4065-a11b-9fde29b3c390 None admin] PATH_INFO: `/v3/users/c62c262317454775845cb45d452c8ab2` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:19.619259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e83ab-4282-4065-a11b-9fde29b3c390 None admin] RBAC: Authorizing `identity:get_user(user_id=c62c262317454775845cb45d452c8ab2)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:19.620952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e83ab-4282-4065-a11b-9fde29b3c390 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:19.622915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 73/153] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:19 2022] GET /identity/v3/users/c62c262317454775845cb45d452c8ab2 => generated 247 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:19.632924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e83ab-4282-4065-a11b-9fde29b3c390 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:19.636321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dc881-dc0a-4972-9219-1246fd1dc5bf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:19.664209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:19.745303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e460edf-6685-458f-bf1f-1598cadfa386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:19.746401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e460edf-6685-458f-bf1f-1598cadfa386 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:19.746891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e460edf-6685-458f-bf1f-1598cadfa386 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:19.747363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e460edf-6685-458f-bf1f-1598cadfa386 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:19.748240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e460edf-6685-458f-bf1f-1598cadfa386 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:19.750377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e460edf-6685-458f-bf1f-1598cadfa386 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:19.779678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e83ab-4282-4065-a11b-9fde29b3c390 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:19.788985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 81/154] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:42:19 2022] POST /identity/v3/users => generated 283 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:42:19.863671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3b19c-00c6-4821-a203-313f4d19cfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:19.864317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3b19c-00c6-4821-a203-313f4d19cfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:19.864596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3b19c-00c6-4821-a203-313f4d19cfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:19.864916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3b19c-00c6-4821-a203-313f4d19cfe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:19.937238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190c49f-d632-4422-8c52-d930677f26dd None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:19.937880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190c49f-d632-4422-8c52-d930677f26dd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:19.938369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190c49f-d632-4422-8c52-d930677f26dd None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:19.939939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 82/155] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:19.945493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cbdd0-7b1f-4f36-81c6-8bbc295b63a5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:19.946564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cbdd0-7b1f-4f36-81c6-8bbc295b63a5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:19.947044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cbdd0-7b1f-4f36-81c6-8bbc295b63a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:19.977578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-825cbdd0-7b1f-4f36-81c6-8bbc295b63a5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:19.979704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-825cbdd0-7b1f-4f36-81c6-8bbc295b63a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:19.996092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3b19c-00c6-4821-a203-313f4d19cfe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:19.997735 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3b19c-00c6-4821-a203-313f4d19cfe5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:20.040031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:20.076162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 83/156] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:20.132677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 74/157] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 10:42:20.134416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825cbdd0-7b1f-4f36-81c6-8bbc295b63a5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:20.168363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef140ff-79cc-467f-ac80-938057332431 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:20.168988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef140ff-79cc-467f-ac80-938057332431 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:20.169478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef140ff-79cc-467f-ac80-938057332431 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:20.171018 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 75/158] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:20.176348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03f372-550d-4be9-b9fc-f7b73fc8daf8 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:20.176937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03f372-550d-4be9-b9fc-f7b73fc8daf8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:20.177405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03f372-550d-4be9-b9fc-f7b73fc8daf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:20.193716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:20.200554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9a03f372-550d-4be9-b9fc-f7b73fc8daf8 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:20.202099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9a03f372-550d-4be9-b9fc-f7b73fc8daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:20.261612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825cbdd0-7b1f-4f36-81c6-8bbc295b63a5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:20.282445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 76/159] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:20.311962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684efde-fe70-4cb1-8cfe-158eb0d9ea73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:20.312527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684efde-fe70-4cb1-8cfe-158eb0d9ea73 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:20.312841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684efde-fe70-4cb1-8cfe-158eb0d9ea73 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:20.313107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684efde-fe70-4cb1-8cfe-158eb0d9ea73 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:20.314240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684efde-fe70-4cb1-8cfe-158eb0d9ea73 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:20.316395 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684efde-fe70-4cb1-8cfe-158eb0d9ea73 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:20.318253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 84/160] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:20 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:20.324206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03f372-550d-4be9-b9fc-f7b73fc8daf8 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:20.394353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:20.404774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684efde-fe70-4cb1-8cfe-158eb0d9ea73 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:20.459360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03f372-550d-4be9-b9fc-f7b73fc8daf8 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:20.505914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fab2f4-e9c7-4874-ab6f-a7e2c150ed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:20.506647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fab2f4-e9c7-4874-ab6f-a7e2c150ed36 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:20.506993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fab2f4-e9c7-4874-ab6f-a7e2c150ed36 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:20.507333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fab2f4-e9c7-4874-ab6f-a7e2c150ed36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:20.508133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fab2f4-e9c7-4874-ab6f-a7e2c150ed36 None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3, user.id=c62c262317454775845cb45d452c8ab2) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:20.508504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fab2f4-e9c7-4874-ab6f-a7e2c150ed36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:20.510086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fab2f4-e9c7-4874-ab6f-a7e2c150ed36 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:20.518082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 77/161] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:20 2022] GET /identity/v3/role_assignments?user.id=c62c262317454775845cb45d452c8ab2&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=0be9777002c641b4a47f78df647cc173 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:20.540949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684efde-fe70-4cb1-8cfe-158eb0d9ea73 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:20.549057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b05437-6f2a-4d46-ae3d-ce57b38eccf1 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:20.550217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b05437-6f2a-4d46-ae3d-ce57b38eccf1 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:20.550780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b05437-6f2a-4d46-ae3d-ce57b38eccf1 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:20.552533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 78/162] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:42:20.562542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaaad7-a887-4b0c-9be4-43086a845405 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:20.563260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaaad7-a887-4b0c-9be4-43086a845405 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:20.563726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaaad7-a887-4b0c-9be4-43086a845405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:20.572566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:20.589910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2caaaad7-a887-4b0c-9be4-43086a845405 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:20.591548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2caaaad7-a887-4b0c-9be4-43086a845405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:20.596629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f9a47-a6dd-4351-8af5-ee87aa963436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:20.597633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9a47-a6dd-4351-8af5-ee87aa963436 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:20.597917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9a47-a6dd-4351-8af5-ee87aa963436 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:20.598310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9a47-a6dd-4351-8af5-ee87aa963436 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:20.614558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f9a47-a6dd-4351-8af5-ee87aa963436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5f8f9a47-a6dd-4351-8af5-ee87aa963436 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:20.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:20.621023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:20.621878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 85/163] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:20.633840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f9a47-a6dd-4351-8af5-ee87aa963436 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:20.700277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 79/164] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:20.701718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaaad7-a887-4b0c-9be4-43086a845405 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:20.787559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:20.804422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f9a47-a6dd-4351-8af5-ee87aa963436 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:20.851320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aff3b-1cfd-468c-bc33-8d5806ab2ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:20.852321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aff3b-1cfd-468c-bc33-8d5806ab2ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:20.853331 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aff3b-1cfd-468c-bc33-8d5806ab2ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:20.854090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aff3b-1cfd-468c-bc33-8d5806ab2ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:20.855192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aff3b-1cfd-468c-bc33-8d5806ab2ea0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:20.855668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aff3b-1cfd-468c-bc33-8d5806ab2ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:20.857920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aff3b-1cfd-468c-bc33-8d5806ab2ea0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:20.858894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaaad7-a887-4b0c-9be4-43086a845405 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:20.868187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 86/165] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:20.873867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aff3b-1cfd-468c-bc33-8d5806ab2ea0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:20.900412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4ba4d-55f4-4a7a-8684-1064b8ddc934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:20.901481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4ba4d-55f4-4a7a-8684-1064b8ddc934 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:20.901931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4ba4d-55f4-4a7a-8684-1064b8ddc934 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:20.902431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4ba4d-55f4-4a7a-8684-1064b8ddc934 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:20.914543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4ba4d-55f4-4a7a-8684-1064b8ddc934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-f7b4ba4d-55f4-4a7a-8684-1064b8ddc934 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:20.919974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:20.921037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:20.922063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 80/166] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:20.926201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:20.929552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4ba4d-55f4-4a7a-8684-1064b8ddc934 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:21.026533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aff3b-1cfd-468c-bc33-8d5806ab2ea0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:21.072192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf3949-93ef-43c2-890d-e5584fc96577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:21.073863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf3949-93ef-43c2-890d-e5584fc96577 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:21.074353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf3949-93ef-43c2-890d-e5584fc96577 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:21.074798 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf3949-93ef-43c2-890d-e5584fc96577 None admin] PATH_INFO: `/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:21.076405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf3949-93ef-43c2-890d-e5584fc96577 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a4ccc7e7914c20a0f1117adfd425c9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:21.078996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf3949-93ef-43c2-890d-e5584fc96577 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:21.079942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4ba4d-55f4-4a7a-8684-1064b8ddc934 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:21.081985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 87/167] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:20 2022] GET /identity/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:21.091737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf3949-93ef-43c2-890d-e5584fc96577 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:21.095305 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:21.135901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33463f99-d0e2-4eb3-b17d-05f76d82d2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:21.136518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33463f99-d0e2-4eb3-b17d-05f76d82d2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:21.136924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33463f99-d0e2-4eb3-b17d-05f76d82d2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:21.137252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33463f99-d0e2-4eb3-b17d-05f76d82d2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:21.138052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33463f99-d0e2-4eb3-b17d-05f76d82d2f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:21.138444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33463f99-d0e2-4eb3-b17d-05f76d82d2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:21.139994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33463f99-d0e2-4eb3-b17d-05f76d82d2f6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:21.149915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 81/168] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:21.150903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33463f99-d0e2-4eb3-b17d-05f76d82d2f6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:21.247982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf3949-93ef-43c2-890d-e5584fc96577 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:21.269273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:21.286413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33463f99-d0e2-4eb3-b17d-05f76d82d2f6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:21.291702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45875c97-ca10-4bf8-92d6-7005b7c7c9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:21.292523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45875c97-ca10-4bf8-92d6-7005b7c7c9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:21.293103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45875c97-ca10-4bf8-92d6-7005b7c7c9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:21.293564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45875c97-ca10-4bf8-92d6-7005b7c7c9a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:21.323700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685e391-fac7-4846-a34a-387c39ae2acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:21.324441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685e391-fac7-4846-a34a-387c39ae2acd None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:21.324786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685e391-fac7-4846-a34a-387c39ae2acd None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:21.325165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685e391-fac7-4846-a34a-387c39ae2acd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:21.337207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685e391-fac7-4846-a34a-387c39ae2acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-f685e391-fac7-4846-a34a-387c39ae2acd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:21.343245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:21.344388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:21.345364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 82/169] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:21.346044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685e391-fac7-4846-a34a-387c39ae2acd None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:21.389040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45875c97-ca10-4bf8-92d6-7005b7c7c9a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:21.391589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45875c97-ca10-4bf8-92d6-7005b7c7c9a3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:21.409869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:21.517359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 88/170] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:21.518676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45875c97-ca10-4bf8-92d6-7005b7c7c9a3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:21.521507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:21.587082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685e391-fac7-4846-a34a-387c39ae2acd None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:21.639890 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45875c97-ca10-4bf8-92d6-7005b7c7c9a3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:21.679629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7f87-3745-49c2-8de9-1480e662abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:21.680761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7f87-3745-49c2-8de9-1480e662abf5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:21.681423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7f87-3745-49c2-8de9-1480e662abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:21.681891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7f87-3745-49c2-8de9-1480e662abf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:21.682748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e7f87-3745-49c2-8de9-1480e662abf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:21.683574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e7f87-3745-49c2-8de9-1480e662abf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:21.685226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e7f87-3745-49c2-8de9-1480e662abf5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:21.695658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 89/171] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:21.696651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07009660-332a-4e61-884b-9d2fa2fca49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:21.697086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7f87-3745-49c2-8de9-1480e662abf5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:21.697442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009660-332a-4e61-884b-9d2fa2fca49c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:21.697725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009660-332a-4e61-884b-9d2fa2fca49c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:21.698021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009660-332a-4e61-884b-9d2fa2fca49c None admin] PATH_INFO: `/v3/users/1e91f8a4248f4ad7886e2a17ed75a4d4` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:21.703785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009660-332a-4e61-884b-9d2fa2fca49c None admin] RBAC: Authorizing `identity:get_user(user_id=1e91f8a4248f4ad7886e2a17ed75a4d4)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:21.705538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009660-332a-4e61-884b-9d2fa2fca49c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:21.711716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 83/172] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:21 2022] GET /identity/v3/users/1e91f8a4248f4ad7886e2a17ed75a4d4 => generated 289 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:21.711716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07009660-332a-4e61-884b-9d2fa2fca49c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:21.789812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:21.838364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7f87-3745-49c2-8de9-1480e662abf5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:21.879165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88052c5-c9f9-46ae-8fa7-0406071c3ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:21.880121 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88052c5-c9f9-46ae-8fa7-0406071c3ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:21.881314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88052c5-c9f9-46ae-8fa7-0406071c3ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:21.881805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88052c5-c9f9-46ae-8fa7-0406071c3ee5 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:21.883181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88052c5-c9f9-46ae-8fa7-0406071c3ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:21.884865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88052c5-c9f9-46ae-8fa7-0406071c3ee5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:21.887269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 90/173] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:21 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:21.888338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88052c5-c9f9-46ae-8fa7-0406071c3ee5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:21.936472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:21.942912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07009660-332a-4e61-884b-9d2fa2fca49c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:22.014791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88052c5-c9f9-46ae-8fa7-0406071c3ee5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:22.026406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2f33c-24d7-4607-a8d7-0ac5a391ee49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:22.027133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2f33c-24d7-4607-a8d7-0ac5a391ee49 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:22.027415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2f33c-24d7-4607-a8d7-0ac5a391ee49 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:22.027687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2f33c-24d7-4607-a8d7-0ac5a391ee49 None admin] PATH_INFO: `/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:22.030776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2f33c-24d7-4607-a8d7-0ac5a391ee49 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a4ccc7e7914c20a0f1117adfd425c9)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:22.032420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2f33c-24d7-4607-a8d7-0ac5a391ee49 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:22.035110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 84/174] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:21 2022] GET /identity/v3/users/c7a4ccc7e7914c20a0f1117adfd425c9 => generated 275 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:22.041869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2f33c-24d7-4607-a8d7-0ac5a391ee49 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:22.054079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:22.061874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72afe0-a13f-4c61-b4d9-824df0943494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:22.062840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72afe0-a13f-4c61-b4d9-824df0943494 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:22.063731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72afe0-a13f-4c61-b4d9-824df0943494 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:22.064213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72afe0-a13f-4c61-b4d9-824df0943494 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:22.067041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72afe0-a13f-4c61-b4d9-824df0943494 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:22.068857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72afe0-a13f-4c61-b4d9-824df0943494 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:22.071079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 91/175] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:21 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:22.072285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72afe0-a13f-4c61-b4d9-824df0943494 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:22.185871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72afe0-a13f-4c61-b4d9-824df0943494 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:22.223356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2f33c-24d7-4607-a8d7-0ac5a391ee49 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:22.226428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ec603-0425-4478-9a84-080e12742d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:22.227270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ec603-0425-4478-9a84-080e12742d11 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:22.227775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ec603-0425-4478-9a84-080e12742d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:22.228249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ec603-0425-4478-9a84-080e12742d11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:22.298892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:22.303092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ec603-0425-4478-9a84-080e12742d11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:22.304796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ec603-0425-4478-9a84-080e12742d11 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:22.310098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61586f-cd0f-4a37-835d-56d14782c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:22.310721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61586f-cd0f-4a37-835d-56d14782c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:22.310998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61586f-cd0f-4a37-835d-56d14782c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:22.311269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61586f-cd0f-4a37-835d-56d14782c2d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:22.312000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61586f-cd0f-4a37-835d-56d14782c2d9 None admin] RBAC: Adding query filter params (role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3, user.id=c7a4ccc7e7914c20a0f1117adfd425c9) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:22.312300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61586f-cd0f-4a37-835d-56d14782c2d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:22.313835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61586f-cd0f-4a37-835d-56d14782c2d9 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:22.331368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 85/176] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:22 2022] GET /identity/v3/role_assignments?user.id=c7a4ccc7e7914c20a0f1117adfd425c9&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=3518baf98ba54a28a9c79ec287469aff => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:22.332269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61586f-cd0f-4a37-835d-56d14782c2d9 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:22.417952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:22.429157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 92/177] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:22.439550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ec603-0425-4478-9a84-080e12742d11 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:22.560689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:22.565002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ec603-0425-4478-9a84-080e12742d11 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:22.575026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61586f-cd0f-4a37-835d-56d14782c2d9 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:22.607024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c90b7-d950-49c4-9549-f6c790c6883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:22.608088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c90b7-d950-49c4-9549-f6c790c6883f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:22.608575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c90b7-d950-49c4-9549-f6c790c6883f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:22.609110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c90b7-d950-49c4-9549-f6c790c6883f None admin] PATH_INFO: `/v3/projects/7875b9b604ed4c9aa28ac6b5252db272` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:22.610613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c90b7-d950-49c4-9549-f6c790c6883f None admin] RBAC: Authorizing `identity:get_project(project_id=7875b9b604ed4c9aa28ac6b5252db272)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:22.612346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c90b7-d950-49c4-9549-f6c790c6883f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:22.614452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 93/178] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:22 2022] GET /identity/v3/projects/7875b9b604ed4c9aa28ac6b5252db272 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:22.621852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c90b7-d950-49c4-9549-f6c790c6883f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:22.656149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3bcf7-7e2c-47f5-a06e-cac17da5deae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:22.657243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3bcf7-7e2c-47f5-a06e-cac17da5deae None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:22.658744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3bcf7-7e2c-47f5-a06e-cac17da5deae None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:22.659872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3bcf7-7e2c-47f5-a06e-cac17da5deae None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/users/1e91f8a4248f4ad7886e2a17ed75a4d4/roles/fdf0d588fb134183911419bbaa627d22` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:22.665380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3bcf7-7e2c-47f5-a06e-cac17da5deae None admin] RBAC: Authorizing `identity:create_grant(project_id=a9cd5f1606ad462482bd725a3a3537a3, user_id=1e91f8a4248f4ad7886e2a17ed75a4d4, role_id=fdf0d588fb134183911419bbaa627d22)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:22.668763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3bcf7-7e2c-47f5-a06e-cac17da5deae None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:22.698261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 86/179] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:42:22 2022] PUT /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/users/1e91f8a4248f4ad7886e2a17ed75a4d4/roles/fdf0d588fb134183911419bbaa627d22 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:22.774020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c90b7-d950-49c4-9549-f6c790c6883f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:22.812838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf56c4d-7227-4fcd-854a-cfb7a96f1aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:22.813818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf56c4d-7227-4fcd-854a-cfb7a96f1aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:22.814460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf56c4d-7227-4fcd-854a-cfb7a96f1aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:22.814961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf56c4d-7227-4fcd-854a-cfb7a96f1aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:22.816822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf56c4d-7227-4fcd-854a-cfb7a96f1aa2 None admin] RBAC: Adding query filter params (role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=7875b9b604ed4c9aa28ac6b5252db272, user.id=c7a4ccc7e7914c20a0f1117adfd425c9) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:22.817349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf56c4d-7227-4fcd-854a-cfb7a96f1aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:22.819021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf56c4d-7227-4fcd-854a-cfb7a96f1aa2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:22.826236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 94/180] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:22 2022] GET /identity/v3/role_assignments?user.id=c7a4ccc7e7914c20a0f1117adfd425c9&scope.project.id=7875b9b604ed4c9aa28ac6b5252db272&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:23.242053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113c02b-2807-458c-862b-5bd45d88f96b None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:23.243095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113c02b-2807-458c-862b-5bd45d88f96b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:23.243512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113c02b-2807-458c-862b-5bd45d88f96b None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:23.245132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 87/181] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:23.250547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69121c-f848-415a-aa06-bcb182ef0160 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:23.250960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69121c-f848-415a-aa06-bcb182ef0160 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:23.251151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69121c-f848-415a-aa06-bcb182ef0160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:23.276650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3d69121c-f848-415a-aa06-bcb182ef0160 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:23.277937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3d69121c-f848-415a-aa06-bcb182ef0160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:23.314786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:23.368750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 95/182] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:23.600275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f1e7d-9bf5-460e-b2ed-26a7d567c8a3 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:23.600935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f1e7d-9bf5-460e-b2ed-26a7d567c8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:23.601416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f1e7d-9bf5-460e-b2ed-26a7d567c8a3 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:23.602968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 88/183] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:23.608444 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576623a9-fe4c-4b76-a411-8f5e528bd710 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:23.608742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576623a9-fe4c-4b76-a411-8f5e528bd710 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:23.609051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576623a9-fe4c-4b76-a411-8f5e528bd710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:23.628989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-576623a9-fe4c-4b76-a411-8f5e528bd710 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:23.630196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-576623a9-fe4c-4b76-a411-8f5e528bd710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:23.659384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f1e7d-9bf5-460e-b2ed-26a7d567c8a3 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:23.702752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 96/184] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:23.704273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f626d-11ce-4d8e-a541-bd68742afac7 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:23.704555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f626d-11ce-4d8e-a541-bd68742afac7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:23.704850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f626d-11ce-4d8e-a541-bd68742afac7 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:23.706463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 97/185] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:23.711849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bffb07-d4ca-49d2-8e90-faac56bd46f5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:23.712136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bffb07-d4ca-49d2-8e90-faac56bd46f5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:23.712414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bffb07-d4ca-49d2-8e90-faac56bd46f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:23.731080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f9bffb07-d4ca-49d2-8e90-faac56bd46f5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:23.732288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f9bffb07-d4ca-49d2-8e90-faac56bd46f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:23.784338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:23.798755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f1e7d-9bf5-460e-b2ed-26a7d567c8a3 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:23.813239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 98/186] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:23.837134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb413636-0375-4b41-bfa2-c277723b95d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:23.837770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb413636-0375-4b41-bfa2-c277723b95d6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:23.838115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb413636-0375-4b41-bfa2-c277723b95d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:23.838391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb413636-0375-4b41-bfa2-c277723b95d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:23.850020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb413636-0375-4b41-bfa2-c277723b95d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-eb413636-0375-4b41-bfa2-c277723b95d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:23.855098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:23.856022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:23.856860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 89/187] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:23.862358 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bffb07-d4ca-49d2-8e90-faac56bd46f5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:23.890145 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:23.971765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bffb07-d4ca-49d2-8e90-faac56bd46f5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.007133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472786f-5c80-4dae-a118-9fb8376cb926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.007532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb413636-0375-4b41-bfa2-c277723b95d6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.007742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472786f-5c80-4dae-a118-9fb8376cb926 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.008177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472786f-5c80-4dae-a118-9fb8376cb926 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.008459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472786f-5c80-4dae-a118-9fb8376cb926 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.009294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472786f-5c80-4dae-a118-9fb8376cb926 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:24.009580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472786f-5c80-4dae-a118-9fb8376cb926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:24.010988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472786f-5c80-4dae-a118-9fb8376cb926 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:24.020509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 99/188] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:24.026285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472786f-5c80-4dae-a118-9fb8376cb926 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.112256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb413636-0375-4b41-bfa2-c277723b95d6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.117672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:24.135507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472786f-5c80-4dae-a118-9fb8376cb926 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.148789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613d670-2d2d-43b6-a468-50573722e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.150025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613d670-2d2d-43b6-a468-50573722e759 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.150314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613d670-2d2d-43b6-a468-50573722e759 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.150976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613d670-2d2d-43b6-a468-50573722e759 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.162470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613d670-2d2d-43b6-a468-50573722e759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-b613d670-2d2d-43b6-a468-50573722e759 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:24.167834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:24.168711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:24.169605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 90/189] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:24.170274 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613d670-2d2d-43b6-a468-50573722e759 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.176979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943fceb-2711-4e7f-9654-643427c9f189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.177632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943fceb-2711-4e7f-9654-643427c9f189 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.177917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943fceb-2711-4e7f-9654-643427c9f189 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.178194 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943fceb-2711-4e7f-9654-643427c9f189 None admin] PATH_INFO: `/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.186032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:24.193978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943fceb-2711-4e7f-9654-643427c9f189 None admin] RBAC: Authorizing `identity:get_user(user_id=31e7c1cd7a734d75bfaf8d7604dfadf8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:24.195926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943fceb-2711-4e7f-9654-643427c9f189 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:24.197633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 100/190] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:24 2022] GET /identity/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:24.197633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943fceb-2711-4e7f-9654-643427c9f189 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.292433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613d670-2d2d-43b6-a468-50573722e759 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.301674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943fceb-2711-4e7f-9654-643427c9f189 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.331549 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc11bad-6f3e-4744-a514-402d9bdffb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.332182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc11bad-6f3e-4744-a514-402d9bdffb96 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.332511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc11bad-6f3e-4744-a514-402d9bdffb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.332787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc11bad-6f3e-4744-a514-402d9bdffb96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.333922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc11bad-6f3e-4744-a514-402d9bdffb96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:24.335494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc11bad-6f3e-4744-a514-402d9bdffb96 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:24.341760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 91/191] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:24.341887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608fb35-420a-45f3-88cb-4f6cfa8a5131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.341981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc11bad-6f3e-4744-a514-402d9bdffb96 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.342054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608fb35-420a-45f3-88cb-4f6cfa8a5131 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.342054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608fb35-420a-45f3-88cb-4f6cfa8a5131 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.342054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608fb35-420a-45f3-88cb-4f6cfa8a5131 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.342054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608fb35-420a-45f3-88cb-4f6cfa8a5131 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:24.342054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608fb35-420a-45f3-88cb-4f6cfa8a5131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:24.342355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608fb35-420a-45f3-88cb-4f6cfa8a5131 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:24.351787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 101/192] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:24.352570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608fb35-420a-45f3-88cb-4f6cfa8a5131 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.358900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:24.430537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:24.452222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc11bad-6f3e-4744-a514-402d9bdffb96 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.467289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608fb35-420a-45f3-88cb-4f6cfa8a5131 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.488553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea805e07-2256-43c0-9560-08c6b8597494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.489188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea805e07-2256-43c0-9560-08c6b8597494 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.489466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea805e07-2256-43c0-9560-08c6b8597494 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.489733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea805e07-2256-43c0-9560-08c6b8597494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.504726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5620ed-ddd9-4a1a-8b3d-b98d9180200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.505403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5620ed-ddd9-4a1a-8b3d-b98d9180200b None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.505661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5620ed-ddd9-4a1a-8b3d-b98d9180200b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.505937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5620ed-ddd9-4a1a-8b3d-b98d9180200b None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.506697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5620ed-ddd9-4a1a-8b3d-b98d9180200b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:24.508036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5620ed-ddd9-4a1a-8b3d-b98d9180200b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:24.530876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 102/193] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:42:24 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:42:24.531896 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5620ed-ddd9-4a1a-8b3d-b98d9180200b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.582815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea805e07-2256-43c0-9560-08c6b8597494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:24.585576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea805e07-2256-43c0-9560-08c6b8597494 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:24.626965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:24.684581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5620ed-ddd9-4a1a-8b3d-b98d9180200b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.689024 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:24.713492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 92/194] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:24.719437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea805e07-2256-43c0-9560-08c6b8597494 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.738081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935167cf-c49c-44cb-90e6-0da3dd83d222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.738871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935167cf-c49c-44cb-90e6-0da3dd83d222 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.739219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935167cf-c49c-44cb-90e6-0da3dd83d222 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.739601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935167cf-c49c-44cb-90e6-0da3dd83d222 None admin] PATH_INFO: `/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.740849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935167cf-c49c-44cb-90e6-0da3dd83d222 None admin] RBAC: Authorizing `identity:get_user(user_id=31e7c1cd7a734d75bfaf8d7604dfadf8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:24.742348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935167cf-c49c-44cb-90e6-0da3dd83d222 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:24.744786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 103/195] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:24 2022] GET /identity/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:24.750269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935167cf-c49c-44cb-90e6-0da3dd83d222 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.843171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea805e07-2256-43c0-9560-08c6b8597494 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.879317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935167cf-c49c-44cb-90e6-0da3dd83d222 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:24.883482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ea39b-ff58-4940-957a-845e901237e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.884244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ea39b-ff58-4940-957a-845e901237e3 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.884535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ea39b-ff58-4940-957a-845e901237e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.884826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ea39b-ff58-4940-957a-845e901237e3 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.885999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ea39b-ff58-4940-957a-845e901237e3 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:24.887464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ea39b-ff58-4940-957a-845e901237e3 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:24.889308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 93/196] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:24 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:24.896598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ea39b-ff58-4940-957a-845e901237e3 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:24.914188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:24.924255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc278c3e-1e4f-4e09-a497-72a54012ea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:24.925699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278c3e-1e4f-4e09-a497-72a54012ea03 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:24.925987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278c3e-1e4f-4e09-a497-72a54012ea03 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:24.926817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278c3e-1e4f-4e09-a497-72a54012ea03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:24.965134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:25.026778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ea39b-ff58-4940-957a-845e901237e3 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:25.027152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc278c3e-1e4f-4e09-a497-72a54012ea03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:25.028860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc278c3e-1e4f-4e09-a497-72a54012ea03 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:25.069181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8396c52-bfa9-4520-b099-dfcfc2a3a64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:25.069799 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8396c52-bfa9-4520-b099-dfcfc2a3a64f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:25.070083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8396c52-bfa9-4520-b099-dfcfc2a3a64f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:25.070520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8396c52-bfa9-4520-b099-dfcfc2a3a64f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:25.071256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8396c52-bfa9-4520-b099-dfcfc2a3a64f None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=caedb3d1f7f94504824174418c72d3df, user.id=31e7c1cd7a734d75bfaf8d7604dfadf8) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:25.071569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8396c52-bfa9-4520-b099-dfcfc2a3a64f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:25.073066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8396c52-bfa9-4520-b099-dfcfc2a3a64f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:25.079411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 94/197] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:24 2022] GET /identity/v3/role_assignments?user.id=31e7c1cd7a734d75bfaf8d7604dfadf8&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=0be9777002c641b4a47f78df647cc173 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:25.152532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 104/198] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 10:42:25.158144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8396c52-bfa9-4520-b099-dfcfc2a3a64f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:25.296294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8396c52-bfa9-4520-b099-dfcfc2a3a64f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:25.312355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:25.336717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d950e9e-a94a-4731-b27b-a6fa6ec99d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:25.337403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d950e9e-a94a-4731-b27b-a6fa6ec99d5e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:25.337685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d950e9e-a94a-4731-b27b-a6fa6ec99d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:25.337963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d950e9e-a94a-4731-b27b-a6fa6ec99d5e None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:25.339193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d950e9e-a94a-4731-b27b-a6fa6ec99d5e None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:25.340715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d950e9e-a94a-4731-b27b-a6fa6ec99d5e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:25.342550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 95/199] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:25 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:25.348948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc278c3e-1e4f-4e09-a497-72a54012ea03 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:25.381176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:25.393037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16e45f-68af-4c8c-a514-d2b5920af96b None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:25.393437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16e45f-68af-4c8c-a514-d2b5920af96b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:25.393647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16e45f-68af-4c8c-a514-d2b5920af96b None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:25.395104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 96/200] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:25.400204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5b90b-03f3-49ae-b18c-c931c19884ad None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:25.400494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5b90b-03f3-49ae-b18c-c931c19884ad None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:25.401006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5b90b-03f3-49ae-b18c-c931c19884ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:25.420409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-07b5b90b-03f3-49ae-b18c-c931c19884ad None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:25.421660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-07b5b90b-03f3-49ae-b18c-c931c19884ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:25.502388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 97/201] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:25.512737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc278c3e-1e4f-4e09-a497-72a54012ea03 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:25.549124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910041a6-795f-4a5a-af79-1d883c6e7659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:25.550639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910041a6-795f-4a5a-af79-1d883c6e7659 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:25.551283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910041a6-795f-4a5a-af79-1d883c6e7659 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:25.551756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910041a6-795f-4a5a-af79-1d883c6e7659 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:25.552682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910041a6-795f-4a5a-af79-1d883c6e7659 None admin] RBAC: Adding query filter params (role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=caedb3d1f7f94504824174418c72d3df, user.id=31e7c1cd7a734d75bfaf8d7604dfadf8) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:25.553216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910041a6-795f-4a5a-af79-1d883c6e7659 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:25.554888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910041a6-795f-4a5a-af79-1d883c6e7659 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:25.561256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 105/202] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:25 2022] GET /identity/v3/role_assignments?user.id=31e7c1cd7a734d75bfaf8d7604dfadf8&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=3518baf98ba54a28a9c79ec287469aff => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:25.837856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5b90b-03f3-49ae-b18c-c931c19884ad None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:25.949327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5b90b-03f3-49ae-b18c-c931c19884ad None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:25.961382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:25.994196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b91350-e7bb-4614-a8fe-9e522898ae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:25.994857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b91350-e7bb-4614-a8fe-9e522898ae46 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:25.995140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b91350-e7bb-4614-a8fe-9e522898ae46 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:25.995409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b91350-e7bb-4614-a8fe-9e522898ae46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:26.007069 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b91350-e7bb-4614-a8fe-9e522898ae46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-14b91350-e7bb-4614-a8fe-9e522898ae46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:26.012080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:26.013040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:26.013870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:26.017871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 98/203] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:26.024485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910041a6-795f-4a5a-af79-1d883c6e7659 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:26.169840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:26.181315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910041a6-795f-4a5a-af79-1d883c6e7659 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:26.220241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334043ad-0278-4d40-b27f-3361ccba7e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:26.220870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334043ad-0278-4d40-b27f-3361ccba7e9f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:26.221151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334043ad-0278-4d40-b27f-3361ccba7e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:26.221526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334043ad-0278-4d40-b27f-3361ccba7e9f None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:26.222474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334043ad-0278-4d40-b27f-3361ccba7e9f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:26.222775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334043ad-0278-4d40-b27f-3361ccba7e9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:26.224150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334043ad-0278-4d40-b27f-3361ccba7e9f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:26.232525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 106/204] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:26.238629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b91350-e7bb-4614-a8fe-9e522898ae46 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:26.343088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b91350-e7bb-4614-a8fe-9e522898ae46 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:26.378524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287885-3607-4df6-831d-95bd69c0bc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:26.379204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287885-3607-4df6-831d-95bd69c0bc8e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:26.379487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287885-3607-4df6-831d-95bd69c0bc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:26.379759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287885-3607-4df6-831d-95bd69c0bc8e None admin] PATH_INFO: `/v3/users/1e91f8a4248f4ad7886e2a17ed75a4d4` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:26.380989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287885-3607-4df6-831d-95bd69c0bc8e None admin] RBAC: Authorizing `identity:get_user(user_id=1e91f8a4248f4ad7886e2a17ed75a4d4)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:26.382442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287885-3607-4df6-831d-95bd69c0bc8e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:26.384184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 99/205] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:26 2022] GET /identity/v3/users/1e91f8a4248f4ad7886e2a17ed75a4d4 => generated 289 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:26.389562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334043ad-0278-4d40-b27f-3361ccba7e9f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:26.521158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334043ad-0278-4d40-b27f-3361ccba7e9f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:26.573833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52c443-4ba7-4cc6-9d13-cdd21b5c18ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:26.574459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52c443-4ba7-4cc6-9d13-cdd21b5c18ce None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:26.574738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52c443-4ba7-4cc6-9d13-cdd21b5c18ce None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:26.575012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52c443-4ba7-4cc6-9d13-cdd21b5c18ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:26.606398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:26.656579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52c443-4ba7-4cc6-9d13-cdd21b5c18ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:26.662172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52c443-4ba7-4cc6-9d13-cdd21b5c18ce None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:26.783136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 107/206] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:26.789419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287885-3607-4df6-831d-95bd69c0bc8e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:26.856723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:26.940672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287885-3607-4df6-831d-95bd69c0bc8e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:26.988462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75a4f8-aa1d-4b76-9f64-890e20e58227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:26.989129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75a4f8-aa1d-4b76-9f64-890e20e58227 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:26.989412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75a4f8-aa1d-4b76-9f64-890e20e58227 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:26.989690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75a4f8-aa1d-4b76-9f64-890e20e58227 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:26.990856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75a4f8-aa1d-4b76-9f64-890e20e58227 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:26.992352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75a4f8-aa1d-4b76-9f64-890e20e58227 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:26.994188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 100/207] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:26 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:27.000221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52c443-4ba7-4cc6-9d13-cdd21b5c18ce None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:27.105474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:27.166745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52c443-4ba7-4cc6-9d13-cdd21b5c18ce None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:27.214852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578f0be-7eb9-4381-85ba-d1f0c00ece43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:27.215470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578f0be-7eb9-4381-85ba-d1f0c00ece43 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:27.215749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578f0be-7eb9-4381-85ba-d1f0c00ece43 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:27.216019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578f0be-7eb9-4381-85ba-d1f0c00ece43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:27.216746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578f0be-7eb9-4381-85ba-d1f0c00ece43 None admin] RBAC: Adding query filter params (role.id=fdf0d588fb134183911419bbaa627d22, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3, user.id=1e91f8a4248f4ad7886e2a17ed75a4d4) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:27.217156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578f0be-7eb9-4381-85ba-d1f0c00ece43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:27.218601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578f0be-7eb9-4381-85ba-d1f0c00ece43 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:27.231408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 108/208] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:26 2022] GET /identity/v3/role_assignments?user.id=1e91f8a4248f4ad7886e2a17ed75a4d4&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=fdf0d588fb134183911419bbaa627d22 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:27.558020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c798-38a1-4c3b-9e33-109a974bd792 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:27.558431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c798-38a1-4c3b-9e33-109a974bd792 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:27.558620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c798-38a1-4c3b-9e33-109a974bd792 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:27.559936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 101/209] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:27.566570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5ed5-c8d7-48d4-9b06-0a78aadb9933 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:27.567085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5ed5-c8d7-48d4-9b06-0a78aadb9933 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:27.567085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5ed5-c8d7-48d4-9b06-0a78aadb9933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:27.594734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a33b5ed5-c8d7-48d4-9b06-0a78aadb9933 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:27.595968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a33b5ed5-c8d7-48d4-9b06-0a78aadb9933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:27.676119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 109/210] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:27.737788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9cf82-7cc6-4ac9-814f-65a4ab4770b9 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:27.738183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9cf82-7cc6-4ac9-814f-65a4ab4770b9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:27.738537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9cf82-7cc6-4ac9-814f-65a4ab4770b9 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:27.740047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 102/211] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:27.745839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e5630-d801-47c2-925f-991693277af7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:27.746226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e5630-d801-47c2-925f-991693277af7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:27.746578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e5630-d801-47c2-925f-991693277af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:27.767220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dd6e5630-d801-47c2-925f-991693277af7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:27.769407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dd6e5630-d801-47c2-925f-991693277af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:27.830859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:27.862958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 110/212] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:27.884496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00308e6b-73ea-4592-9863-00c1c6ac5fd9 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:27.884840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00308e6b-73ea-4592-9863-00c1c6ac5fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:27.885103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00308e6b-73ea-4592-9863-00c1c6ac5fd9 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:27.886460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 103/213] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:27.891586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a7fcd-371e-4d7a-9dc4-f1eb1568be02 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:27.891879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a7fcd-371e-4d7a-9dc4-f1eb1568be02 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:27.892157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a7fcd-371e-4d7a-9dc4-f1eb1568be02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:27.915825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-454a7fcd-371e-4d7a-9dc4-f1eb1568be02 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:27.917053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-454a7fcd-371e-4d7a-9dc4-f1eb1568be02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:27.989602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00308e6b-73ea-4592-9863-00c1c6ac5fd9 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:27.992320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 111/214] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:28.106647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00308e6b-73ea-4592-9863-00c1c6ac5fd9 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:28.116109 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:28.147342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18715-26a3-4754-9033-cbec4cb05f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:28.148405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18715-26a3-4754-9033-cbec4cb05f0d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.148929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18715-26a3-4754-9033-cbec4cb05f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.149484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18715-26a3-4754-9033-cbec4cb05f0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.150848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a7fcd-371e-4d7a-9dc4-f1eb1568be02 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:28.163313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea18715-26a3-4754-9033-cbec4cb05f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-cea18715-26a3-4754-9033-cbec4cb05f0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:28.169564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:28.170446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:28.171247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:28.171848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 104/215] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:28.177807 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18715-26a3-4754-9033-cbec4cb05f0d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:28.263191 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a7fcd-371e-4d7a-9dc4-f1eb1568be02 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:28.273477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:28.287525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18715-26a3-4754-9033-cbec4cb05f0d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:28.300142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf716459-0446-4be9-88e0-c3a7aee0eab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:28.300771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf716459-0446-4be9-88e0-c3a7aee0eab3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.301094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf716459-0446-4be9-88e0-c3a7aee0eab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.301362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf716459-0446-4be9-88e0-c3a7aee0eab3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.302465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf716459-0446-4be9-88e0-c3a7aee0eab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:28.303949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf716459-0446-4be9-88e0-c3a7aee0eab3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:28.305710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 112/216] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:28.306802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0810c0-d17f-490c-a4e6-a1f1fbfc07cf None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.307087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0810c0-d17f-490c-a4e6-a1f1fbfc07cf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.307357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0810c0-d17f-490c-a4e6-a1f1fbfc07cf None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.308566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 113/217] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:28.312700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0810c0-d17f-490c-a4e6-a1f1fbfc07cf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:28.323015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be767d-0ec8-4534-b5f1-b60019cb613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:28.323627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be767d-0ec8-4534-b5f1-b60019cb613c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.323915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be767d-0ec8-4534-b5f1-b60019cb613c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.324185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be767d-0ec8-4534-b5f1-b60019cb613c None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.324905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be767d-0ec8-4534-b5f1-b60019cb613c None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:28.325212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be767d-0ec8-4534-b5f1-b60019cb613c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:28.326615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be767d-0ec8-4534-b5f1-b60019cb613c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:28.335640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 105/218] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:28.336499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be767d-0ec8-4534-b5f1-b60019cb613c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:28.364333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:28.411768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0810c0-d17f-490c-a4e6-a1f1fbfc07cf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:28.434068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be767d-0ec8-4534-b5f1-b60019cb613c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:28.445272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a0d73-c4a3-4ebf-ab79-8f2625b7ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:28.445901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a0d73-c4a3-4ebf-ab79-8f2625b7ede2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.446176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a0d73-c4a3-4ebf-ab79-8f2625b7ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.446445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a0d73-c4a3-4ebf-ab79-8f2625b7ede2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.457401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a0d73-c4a3-4ebf-ab79-8f2625b7ede2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-7f8a0d73-c4a3-4ebf-ab79-8f2625b7ede2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:28.462390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:28.463302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:28.464133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 114/219] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:42:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:28.465201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052498dd-17be-4991-b695-571bdd521b83 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.465500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052498dd-17be-4991-b695-571bdd521b83 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.465761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052498dd-17be-4991-b695-571bdd521b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.469189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511c894-8179-4819-8d82-cd3bf77082a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:28.469752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511c894-8179-4819-8d82-cd3bf77082a8 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.470034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511c894-8179-4819-8d82-cd3bf77082a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.470310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511c894-8179-4819-8d82-cd3bf77082a8 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.470962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511c894-8179-4819-8d82-cd3bf77082a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:28.472437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511c894-8179-4819-8d82-cd3bf77082a8 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:28.482959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-052498dd-17be-4991-b695-571bdd521b83 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:28.484171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-052498dd-17be-4991-b695-571bdd521b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:28.494394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 106/220] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:42:28 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:42:28.495776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511c894-8179-4819-8d82-cd3bf77082a8 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:28.520605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:28.572985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 115/221] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:28.585842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052498dd-17be-4991-b695-571bdd521b83 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:28.622866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:28.642322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511c894-8179-4819-8d82-cd3bf77082a8 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:28.689851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3f059-5e21-498c-823e-1950d56ac547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:28.691204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3f059-5e21-498c-823e-1950d56ac547 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.691693 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3f059-5e21-498c-823e-1950d56ac547 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.692185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3f059-5e21-498c-823e-1950d56ac547 None admin] PATH_INFO: `/v3/users/be446bcb0abb4e3f96d98ff4b35df27a` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.708064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052498dd-17be-4991-b695-571bdd521b83 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:28.712111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3f059-5e21-498c-823e-1950d56ac547 None admin] RBAC: Authorizing `identity:get_user(user_id=be446bcb0abb4e3f96d98ff4b35df27a)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:28.713814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3f059-5e21-498c-823e-1950d56ac547 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:28.716485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 107/222] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:28 2022] GET /identity/v3/users/be446bcb0abb4e3f96d98ff4b35df27a => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:28.722836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3f059-5e21-498c-823e-1950d56ac547 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:28.750450 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822d6af-5fdc-4d2d-8dad-c9e70c5da211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:28.751079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822d6af-5fdc-4d2d-8dad-c9e70c5da211 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.751362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822d6af-5fdc-4d2d-8dad-c9e70c5da211 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.751634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822d6af-5fdc-4d2d-8dad-c9e70c5da211 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.752928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822d6af-5fdc-4d2d-8dad-c9e70c5da211 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:28.753392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822d6af-5fdc-4d2d-8dad-c9e70c5da211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:28.754804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822d6af-5fdc-4d2d-8dad-c9e70c5da211 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:28.764685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 116/223] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:28.771637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822d6af-5fdc-4d2d-8dad-c9e70c5da211 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:28.802891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:28.858860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3f059-5e21-498c-823e-1950d56ac547 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:28.895781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822d6af-5fdc-4d2d-8dad-c9e70c5da211 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:28.910480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be2c2d-acef-4e31-9597-fc19d4fba404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:28.911096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be2c2d-acef-4e31-9597-fc19d4fba404 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.911378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be2c2d-acef-4e31-9597-fc19d4fba404 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.911706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be2c2d-acef-4e31-9597-fc19d4fba404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.927353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:28.932896 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9639ad9-1337-4a5d-a52d-14ad4f1f67f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:28.934133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9639ad9-1337-4a5d-a52d-14ad4f1f67f0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:28.934420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9639ad9-1337-4a5d-a52d-14ad4f1f67f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:28.934694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9639ad9-1337-4a5d-a52d-14ad4f1f67f0 None admin] PATH_INFO: `/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:28.936338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9639ad9-1337-4a5d-a52d-14ad4f1f67f0 None admin] RBAC: Authorizing `identity:get_user(user_id=31e7c1cd7a734d75bfaf8d7604dfadf8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:28.938716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9639ad9-1337-4a5d-a52d-14ad4f1f67f0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:28.940198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 117/224] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:28 2022] GET /identity/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:28.941332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9639ad9-1337-4a5d-a52d-14ad4f1f67f0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:28.996412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be2c2d-acef-4e31-9597-fc19d4fba404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:28.997940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be2c2d-acef-4e31-9597-fc19d4fba404 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:29.074348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:29.081405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9639ad9-1337-4a5d-a52d-14ad4f1f67f0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:29.120442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 108/225] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:29.124992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be2c2d-acef-4e31-9597-fc19d4fba404 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:29.129003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616370bd-bc34-45a9-b5c9-de4f25cb8fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:29.129655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616370bd-bc34-45a9-b5c9-de4f25cb8fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:29.129938 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616370bd-bc34-45a9-b5c9-de4f25cb8fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:29.130212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616370bd-bc34-45a9-b5c9-de4f25cb8fa9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:29.141797 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616370bd-bc34-45a9-b5c9-de4f25cb8fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-616370bd-bc34-45a9-b5c9-de4f25cb8fa9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:29.149149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:29.150279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 118/226] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:29.151276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616370bd-bc34-45a9-b5c9-de4f25cb8fa9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:29.174372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:29.248611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be2c2d-acef-4e31-9597-fc19d4fba404 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:29.290081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616370bd-bc34-45a9-b5c9-de4f25cb8fa9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:29.297693 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3671c0-4559-4075-ba3b-59c8996ab00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:29.298331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3671c0-4559-4075-ba3b-59c8996ab00d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:29.298614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3671c0-4559-4075-ba3b-59c8996ab00d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:29.299460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3671c0-4559-4075-ba3b-59c8996ab00d None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:29.300732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3671c0-4559-4075-ba3b-59c8996ab00d None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:29.302751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3671c0-4559-4075-ba3b-59c8996ab00d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:29.305192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 109/227] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:29 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:29.306399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3671c0-4559-4075-ba3b-59c8996ab00d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:29.334682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd96f46-76dd-4157-9b8e-c4b7ce46a893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:29.335326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd96f46-76dd-4157-9b8e-c4b7ce46a893 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:29.336150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd96f46-76dd-4157-9b8e-c4b7ce46a893 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:29.336426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd96f46-76dd-4157-9b8e-c4b7ce46a893 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:29.337998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd96f46-76dd-4157-9b8e-c4b7ce46a893 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:29.340112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd96f46-76dd-4157-9b8e-c4b7ce46a893 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:29.342427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 119/228] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:29 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:29.343162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd96f46-76dd-4157-9b8e-c4b7ce46a893 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:29.377823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:29.445560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3671c0-4559-4075-ba3b-59c8996ab00d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:29.461057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:29.472491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd96f46-76dd-4157-9b8e-c4b7ce46a893 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:29.494874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ff540-1a23-40fb-8cf0-ac3cddb1e9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:29.496054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ff540-1a23-40fb-8cf0-ac3cddb1e9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:29.496339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ff540-1a23-40fb-8cf0-ac3cddb1e9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:29.496620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ff540-1a23-40fb-8cf0-ac3cddb1e9a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:29.497617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ff540-1a23-40fb-8cf0-ac3cddb1e9a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:29.497927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ff540-1a23-40fb-8cf0-ac3cddb1e9a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:29.500304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ff540-1a23-40fb-8cf0-ac3cddb1e9a8 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:29.510862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b87cef-1524-424d-a5dc-14f83daf0b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:29.511388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 110/229] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:29.511674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b87cef-1524-424d-a5dc-14f83daf0b95 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:29.512018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b87cef-1524-424d-a5dc-14f83daf0b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:29.512212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ff540-1a23-40fb-8cf0-ac3cddb1e9a8 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:29.512378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b87cef-1524-424d-a5dc-14f83daf0b95 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df/users/31e7c1cd7a734d75bfaf8d7604dfadf8/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:29.514412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b87cef-1524-424d-a5dc-14f83daf0b95 None admin] RBAC: Authorizing `identity:create_grant(project_id=caedb3d1f7f94504824174418c72d3df, user_id=31e7c1cd7a734d75bfaf8d7604dfadf8, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:29.516478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b87cef-1524-424d-a5dc-14f83daf0b95 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:29.532695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 120/230] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:42:29 2022] PUT /identity/v3/projects/caedb3d1f7f94504824174418c72d3df/users/31e7c1cd7a734d75bfaf8d7604dfadf8/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:29.534450 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b87cef-1524-424d-a5dc-14f83daf0b95 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:29.641666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:29.651662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ff540-1a23-40fb-8cf0-ac3cddb1e9a8 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:29.664891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b87cef-1524-424d-a5dc-14f83daf0b95 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:29.702774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4b76b-0850-4c99-b422-b841ba80f094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:29.706126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4b76b-0850-4c99-b422-b841ba80f094 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:29.706847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4b76b-0850-4c99-b422-b841ba80f094 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:29.708142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4b76b-0850-4c99-b422-b841ba80f094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:29.709124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4b76b-0850-4c99-b422-b841ba80f094 None admin] RBAC: Adding query filter params (role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=caedb3d1f7f94504824174418c72d3df, user.id=be446bcb0abb4e3f96d98ff4b35df27a) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:29.710842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4b76b-0850-4c99-b422-b841ba80f094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:29.713139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4b76b-0850-4c99-b422-b841ba80f094 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:29.718900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b78303-36e6-494b-a194-10bdca196b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:29.719777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b78303-36e6-494b-a194-10bdca196b8a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:29.720198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b78303-36e6-494b-a194-10bdca196b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:29.720609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b78303-36e6-494b-a194-10bdca196b8a None admin] PATH_INFO: `/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:29.724222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b78303-36e6-494b-a194-10bdca196b8a None admin] RBAC: Authorizing `identity:get_user(user_id=31e7c1cd7a734d75bfaf8d7604dfadf8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:29.726143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 111/231] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:29 2022] GET /identity/v3/role_assignments?user.id=be446bcb0abb4e3f96d98ff4b35df27a&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:29.733146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b78303-36e6-494b-a194-10bdca196b8a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:29.733146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 121/232] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:29 2022] GET /identity/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:29.739774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4b76b-0850-4c99-b422-b841ba80f094 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:29.794756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:29.872614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4b76b-0850-4c99-b422-b841ba80f094 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:29.915703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b086f-adf7-4cdb-91d7-5fad0241fe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:29.916840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b086f-adf7-4cdb-91d7-5fad0241fe74 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:29.917335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b086f-adf7-4cdb-91d7-5fad0241fe74 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:29.917840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b086f-adf7-4cdb-91d7-5fad0241fe74 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:29.919586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b086f-adf7-4cdb-91d7-5fad0241fe74 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:29.921499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b086f-adf7-4cdb-91d7-5fad0241fe74 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:29.923580 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 112/233] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:29 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:29.931394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b78303-36e6-494b-a194-10bdca196b8a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:30.062260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b78303-36e6-494b-a194-10bdca196b8a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:30.110058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d871488-ded3-4f5d-9c41-e2ce754dd143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:30.110734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871488-ded3-4f5d-9c41-e2ce754dd143 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:30.111012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871488-ded3-4f5d-9c41-e2ce754dd143 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:30.111281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871488-ded3-4f5d-9c41-e2ce754dd143 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df/users/31e7c1cd7a734d75bfaf8d7604dfadf8/roles/3518baf98ba54a28a9c79ec287469aff` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:30.114479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d871488-ded3-4f5d-9c41-e2ce754dd143 None admin] RBAC: Authorizing `identity:create_grant(project_id=caedb3d1f7f94504824174418c72d3df, user_id=31e7c1cd7a734d75bfaf8d7604dfadf8, role_id=3518baf98ba54a28a9c79ec287469aff)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:30.115946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d871488-ded3-4f5d-9c41-e2ce754dd143 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:30.147552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 122/234] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:42:29 2022] PUT /identity/v3/projects/caedb3d1f7f94504824174418c72d3df/users/31e7c1cd7a734d75bfaf8d7604dfadf8/roles/3518baf98ba54a28a9c79ec287469aff => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:31.293699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d770a-405e-474f-a8ea-7ba3d5146026 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:31.294248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d770a-405e-474f-a8ea-7ba3d5146026 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:31.294248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d770a-405e-474f-a8ea-7ba3d5146026 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:31.295603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 113/235] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:31.301232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac44fa-4851-4a1f-92f6-10b806858d47 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:31.302025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac44fa-4851-4a1f-92f6-10b806858d47 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:31.302497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac44fa-4851-4a1f-92f6-10b806858d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:31.325103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e3ac44fa-4851-4a1f-92f6-10b806858d47 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:31.326633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e3ac44fa-4851-4a1f-92f6-10b806858d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:31.338804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:31.418609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 123/236] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:31.605035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378046da-4fd9-4e0f-a2d0-53e2b5886ca0 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:31.605325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378046da-4fd9-4e0f-a2d0-53e2b5886ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:31.605773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378046da-4fd9-4e0f-a2d0-53e2b5886ca0 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:31.607336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 114/237] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:31.613729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f99fd4-b8bf-4190-a01a-73f62aa86043 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:31.614018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f99fd4-b8bf-4190-a01a-73f62aa86043 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:31.614290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f99fd4-b8bf-4190-a01a-73f62aa86043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:31.637736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-00f99fd4-b8bf-4190-a01a-73f62aa86043 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:31.638900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-00f99fd4-b8bf-4190-a01a-73f62aa86043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:31.731417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 124/238] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:31.890107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378046da-4fd9-4e0f-a2d0-53e2b5886ca0 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:32.035065 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378046da-4fd9-4e0f-a2d0-53e2b5886ca0 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:32.037570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:32.075779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b5b8c-bb95-4d15-a09d-b91aa381061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:32.076418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b5b8c-bb95-4d15-a09d-b91aa381061f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.076692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b5b8c-bb95-4d15-a09d-b91aa381061f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.076988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b5b8c-bb95-4d15-a09d-b91aa381061f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.088060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b5b8c-bb95-4d15-a09d-b91aa381061f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-4f6b5b8c-bb95-4d15-a09d-b91aa381061f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:32.092963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:32.093886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:32.094689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 115/239] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:32.105914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f99fd4-b8bf-4190-a01a-73f62aa86043 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:32.136022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b5b8c-bb95-4d15-a09d-b91aa381061f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:32.264020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b5b8c-bb95-4d15-a09d-b91aa381061f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:32.292981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:32.306881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224ce4e-44c9-4bfb-b535-760462176772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:32.308323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224ce4e-44c9-4bfb-b535-760462176772 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.308788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224ce4e-44c9-4bfb-b535-760462176772 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.309276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224ce4e-44c9-4bfb-b535-760462176772 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.310709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224ce4e-44c9-4bfb-b535-760462176772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:32.312473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224ce4e-44c9-4bfb-b535-760462176772 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:32.317544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 116/240] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:32.320910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224ce4e-44c9-4bfb-b535-760462176772 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:32.354315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f99fd4-b8bf-4190-a01a-73f62aa86043 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:32.366516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:32.403363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e8c89-e695-4af4-8750-5be97a8f0b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:32.404310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e8c89-e695-4af4-8750-5be97a8f0b69 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.404975 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e8c89-e695-4af4-8750-5be97a8f0b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.405780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e8c89-e695-4af4-8750-5be97a8f0b69 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.406908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e8c89-e695-4af4-8750-5be97a8f0b69 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:32.407394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e8c89-e695-4af4-8750-5be97a8f0b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:32.409747 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e8c89-e695-4af4-8750-5be97a8f0b69 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:32.420615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 125/241] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:32.422305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1af9bc-0194-4272-965b-49a7f32805e4 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.422779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1af9bc-0194-4272-965b-49a7f32805e4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.423233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1af9bc-0194-4272-965b-49a7f32805e4 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.425197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 126/242] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:32.426601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1af9bc-0194-4272-965b-49a7f32805e4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:32.458293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224ce4e-44c9-4bfb-b535-760462176772 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:32.498122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9bdb-3184-4251-abbf-ec01d962767d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:32.499140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9bdb-3184-4251-abbf-ec01d962767d None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.499427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9bdb-3184-4251-abbf-ec01d962767d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.499696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9bdb-3184-4251-abbf-ec01d962767d None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.500619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9bdb-3184-4251-abbf-ec01d962767d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:32.502837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9bdb-3184-4251-abbf-ec01d962767d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:32.528710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 117/243] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:42:32 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:42:32.539533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777cb86-dbb1-4618-80f8-7f1d6a4d2456 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.540002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777cb86-dbb1-4618-80f8-7f1d6a4d2456 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.540422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777cb86-dbb1-4618-80f8-7f1d6a4d2456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.554634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1af9bc-0194-4272-965b-49a7f32805e4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:32.572986 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3777cb86-dbb1-4618-80f8-7f1d6a4d2456 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:32.574702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3777cb86-dbb1-4618-80f8-7f1d6a4d2456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:32.595219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93782155-5f8b-4b28-805d-1bc7c9c8bb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:32.596955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93782155-5f8b-4b28-805d-1bc7c9c8bb0b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.597427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93782155-5f8b-4b28-805d-1bc7c9c8bb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.598006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93782155-5f8b-4b28-805d-1bc7c9c8bb0b None admin] PATH_INFO: `/v3/users/216176a23a44431f934c25ebac99ecee` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.600758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:32.615997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93782155-5f8b-4b28-805d-1bc7c9c8bb0b None admin] RBAC: Authorizing `identity:get_user(user_id=216176a23a44431f934c25ebac99ecee)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:32.618022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93782155-5f8b-4b28-805d-1bc7c9c8bb0b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:32.624865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 127/244] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:32 2022] GET /identity/v3/users/216176a23a44431f934c25ebac99ecee => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:32.626844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d0bdd-ff8d-45d5-8aa6-48e5937ed463 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.627173 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d0bdd-ff8d-45d5-8aa6-48e5937ed463 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.627454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d0bdd-ff8d-45d5-8aa6-48e5937ed463 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.629248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 128/245] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:32.630036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d0bdd-ff8d-45d5-8aa6-48e5937ed463 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:32.674216 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 118/246] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:32.681241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e2c94-180c-4250-9c64-b75f3416b32e None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.681241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e2c94-180c-4250-9c64-b75f3416b32e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.681241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e2c94-180c-4250-9c64-b75f3416b32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.716248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-fa5e2c94-180c-4250-9c64-b75f3416b32e None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:32.720541 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fa5e2c94-180c-4250-9c64-b75f3416b32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:32.747810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:32.796187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d0bdd-ff8d-45d5-8aa6-48e5937ed463 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:32.810892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 119/247] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:32.846933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f1dc9-af25-4197-9b6d-67eeb293832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:32.847563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f1dc9-af25-4197-9b6d-67eeb293832b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:32.847848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f1dc9-af25-4197-9b6d-67eeb293832b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:32.848125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f1dc9-af25-4197-9b6d-67eeb293832b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:32.918351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:32.931380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f1dc9-af25-4197-9b6d-67eeb293832b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:32.932901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f1dc9-af25-4197-9b6d-67eeb293832b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:32.995708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e2c94-180c-4250-9c64-b75f3416b32e None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:33.084858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 129/248] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:33.090550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f1dc9-af25-4197-9b6d-67eeb293832b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:33.153405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e2c94-180c-4250-9c64-b75f3416b32e None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:33.197985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f6721-7665-4b04-bee4-4edbc8a53746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:33.198884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f6721-7665-4b04-bee4-4edbc8a53746 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:33.199173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f6721-7665-4b04-bee4-4edbc8a53746 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:33.199447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f6721-7665-4b04-bee4-4edbc8a53746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:33.209557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:33.214881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f6721-7665-4b04-bee4-4edbc8a53746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:33.220352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f1dc9-af25-4197-9b6d-67eeb293832b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-d00f6721-7665-4b04-bee4-4edbc8a53746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:33.220478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:33.221802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:33.222687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 120/249] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:42:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:33.226341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a6c18-3783-42b3-9e36-270f1627f0bd None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:33.226341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a6c18-3783-42b3-9e36-270f1627f0bd None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:33.226341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a6c18-3783-42b3-9e36-270f1627f0bd None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:33.226341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 121/250] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:33.226895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a6c18-3783-42b3-9e36-270f1627f0bd None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:33.241053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:33.273101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b4ad4-63e8-4e7f-93d4-e6de4e316d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:33.274337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b4ad4-63e8-4e7f-93d4-e6de4e316d39 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:33.274817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b4ad4-63e8-4e7f-93d4-e6de4e316d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:33.276236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b4ad4-63e8-4e7f-93d4-e6de4e316d39 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:33.277620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b4ad4-63e8-4e7f-93d4-e6de4e316d39 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:33.279296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b4ad4-63e8-4e7f-93d4-e6de4e316d39 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:33.281828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 130/251] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:33 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:33.282805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b4ad4-63e8-4e7f-93d4-e6de4e316d39 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:33.363350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a6c18-3783-42b3-9e36-270f1627f0bd None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:33.403833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b4ad4-63e8-4e7f-93d4-e6de4e316d39 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:33.410886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bae2ac-7f21-4f18-8a05-d1a157f3f974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:33.411518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bae2ac-7f21-4f18-8a05-d1a157f3f974 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:33.411798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bae2ac-7f21-4f18-8a05-d1a157f3f974 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:33.412069 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bae2ac-7f21-4f18-8a05-d1a157f3f974 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:33.426249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bae2ac-7f21-4f18-8a05-d1a157f3f974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-53bae2ac-7f21-4f18-8a05-d1a157f3f974 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:33.432384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:33.433333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:33.434131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 122/252] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:33.437323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d5a2b-2d74-4c6c-9e6a-65e3ed9c089e None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:33.437323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d5a2b-2d74-4c6c-9e6a-65e3ed9c089e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:33.437323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d5a2b-2d74-4c6c-9e6a-65e3ed9c089e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:33.450949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6de081-8e46-4fc8-9136-9f2fbd3fa689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:33.451957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6de081-8e46-4fc8-9136-9f2fbd3fa689 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:33.452487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6de081-8e46-4fc8-9136-9f2fbd3fa689 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:33.453265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6de081-8e46-4fc8-9136-9f2fbd3fa689 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:33.454131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6de081-8e46-4fc8-9136-9f2fbd3fa689 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:33.454628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6de081-8e46-4fc8-9136-9f2fbd3fa689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:33.456866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6de081-8e46-4fc8-9136-9f2fbd3fa689 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:33.457782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-511d5a2b-2d74-4c6c-9e6a-65e3ed9c089e None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:33.459263 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-511d5a2b-2d74-4c6c-9e6a-65e3ed9c089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:33.466542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 131/253] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:33.467614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6de081-8e46-4fc8-9136-9f2fbd3fa689 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:33.494504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:33.536689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:33.552829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 123/254] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:33.552829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d5a2b-2d74-4c6c-9e6a-65e3ed9c089e None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:33.623156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6de081-8e46-4fc8-9136-9f2fbd3fa689 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:33.660586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d5a2b-2d74-4c6c-9e6a-65e3ed9c089e None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:33.698365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e1b1a-c181-46fd-9409-bf9d7b3ada7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:33.698989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e1b1a-c181-46fd-9409-bf9d7b3ada7e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:33.699277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e1b1a-c181-46fd-9409-bf9d7b3ada7e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:33.699551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e1b1a-c181-46fd-9409-bf9d7b3ada7e None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:33.700223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e1b1a-c181-46fd-9409-bf9d7b3ada7e None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:33.700529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e1b1a-c181-46fd-9409-bf9d7b3ada7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:33.701948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e1b1a-c181-46fd-9409-bf9d7b3ada7e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:33.706156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982c8ed-2975-4691-b297-e9289e05d5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:33.706778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982c8ed-2975-4691-b297-e9289e05d5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:33.707056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982c8ed-2975-4691-b297-e9289e05d5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:33.707328 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982c8ed-2975-4691-b297-e9289e05d5b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:33.708057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982c8ed-2975-4691-b297-e9289e05d5b1 None admin] RBAC: Adding query filter params (role.id=fdf0d588fb134183911419bbaa627d22, scope.project.id=caedb3d1f7f94504824174418c72d3df, user.id=216176a23a44431f934c25ebac99ecee) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:33.708367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982c8ed-2975-4691-b297-e9289e05d5b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:33.709856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982c8ed-2975-4691-b297-e9289e05d5b1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:33.722259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 132/255] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:33 2022] GET /identity/v3/role_assignments?user.id=216176a23a44431f934c25ebac99ecee&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=fdf0d588fb134183911419bbaa627d22 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:33.723102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982c8ed-2975-4691-b297-e9289e05d5b1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:33.725894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 124/256] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:33.733073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e1b1a-c181-46fd-9409-bf9d7b3ada7e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:33.865218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:33.916850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e1b1a-c181-46fd-9409-bf9d7b3ada7e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:33.954591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:33.960082 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982c8ed-2975-4691-b297-e9289e05d5b1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:34.010676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42694c8-ca42-4be8-81c9-6bb5d9de37fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:34.011338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42694c8-ca42-4be8-81c9-6bb5d9de37fd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:34.011623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42694c8-ca42-4be8-81c9-6bb5d9de37fd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:34.011902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42694c8-ca42-4be8-81c9-6bb5d9de37fd None admin] PATH_INFO: `/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:34.013456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42694c8-ca42-4be8-81c9-6bb5d9de37fd None admin] RBAC: Authorizing `identity:get_user(user_id=31e7c1cd7a734d75bfaf8d7604dfadf8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:34.014924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42694c8-ca42-4be8-81c9-6bb5d9de37fd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:34.016696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 133/257] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:33 2022] GET /identity/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8 => generated 283 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:34.021548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42694c8-ca42-4be8-81c9-6bb5d9de37fd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:34.023775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab1175-709c-4876-98e6-164f249b59ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:34.024438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab1175-709c-4876-98e6-164f249b59ba None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:34.024720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab1175-709c-4876-98e6-164f249b59ba None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:34.025020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab1175-709c-4876-98e6-164f249b59ba None admin] PATH_INFO: `/v3/users/be446bcb0abb4e3f96d98ff4b35df27a` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:34.026834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab1175-709c-4876-98e6-164f249b59ba None admin] RBAC: Authorizing `identity:get_user(user_id=be446bcb0abb4e3f96d98ff4b35df27a)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:34.028293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab1175-709c-4876-98e6-164f249b59ba None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:34.030237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 125/258] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:33 2022] GET /identity/v3/users/be446bcb0abb4e3f96d98ff4b35df27a => generated 297 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:34.031305 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab1175-709c-4876-98e6-164f249b59ba None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:34.154149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab1175-709c-4876-98e6-164f249b59ba None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:34.182593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:34.182752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42694c8-ca42-4be8-81c9-6bb5d9de37fd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:34.193318 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195608da-ad29-46f5-8c0f-dd1991a69a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:34.194354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195608da-ad29-46f5-8c0f-dd1991a69a36 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:34.194737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195608da-ad29-46f5-8c0f-dd1991a69a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:34.195116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195608da-ad29-46f5-8c0f-dd1991a69a36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:34.237787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a72605-99e6-4075-b858-b0f62862f5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:34.240376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a72605-99e6-4075-b858-b0f62862f5eb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:34.240827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a72605-99e6-4075-b858-b0f62862f5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:34.241245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a72605-99e6-4075-b858-b0f62862f5eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:34.267654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a72605-99e6-4075-b858-b0f62862f5eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-d3a72605-99e6-4075-b858-b0f62862f5eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:34.273622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:34.274697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:34.275781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:34.284158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 134/259] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:34.285307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a72605-99e6-4075-b858-b0f62862f5eb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:34.326388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195608da-ad29-46f5-8c0f-dd1991a69a36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:34.327874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195608da-ad29-46f5-8c0f-dd1991a69a36 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:34.331093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:34.438252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 126/260] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:34.439042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195608da-ad29-46f5-8c0f-dd1991a69a36 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:34.486005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:34.534971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a72605-99e6-4075-b858-b0f62862f5eb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:34.591547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ec33a-64bf-41c4-a783-66d5ac52163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:34.592610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec33a-64bf-41c4-a783-66d5ac52163f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:34.592923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec33a-64bf-41c4-a783-66d5ac52163f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:34.593207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec33a-64bf-41c4-a783-66d5ac52163f None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:34.594346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ec33a-64bf-41c4-a783-66d5ac52163f None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:34.596038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ec33a-64bf-41c4-a783-66d5ac52163f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:34.598352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 135/261] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:34 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:34.599275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ec33a-64bf-41c4-a783-66d5ac52163f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:34.630159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195608da-ad29-46f5-8c0f-dd1991a69a36 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:34.699389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852060cd-f01a-45be-ba08-75b7baba6329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:34.699929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852060cd-f01a-45be-ba08-75b7baba6329 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:34.700212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852060cd-f01a-45be-ba08-75b7baba6329 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:34.700487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852060cd-f01a-45be-ba08-75b7baba6329 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:34.701230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852060cd-f01a-45be-ba08-75b7baba6329 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:34.701542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852060cd-f01a-45be-ba08-75b7baba6329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:34.702947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852060cd-f01a-45be-ba08-75b7baba6329 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:34.712646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 127/262] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:34.713479 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852060cd-f01a-45be-ba08-75b7baba6329 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:34.741531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:34.773646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ec33a-64bf-41c4-a783-66d5ac52163f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:34.841190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e126268-187e-4f7b-87c7-b2fef5d26bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:34.841829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e126268-187e-4f7b-87c7-b2fef5d26bbc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:34.842105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e126268-187e-4f7b-87c7-b2fef5d26bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:34.842374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e126268-187e-4f7b-87c7-b2fef5d26bbc None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:34.845951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e126268-187e-4f7b-87c7-b2fef5d26bbc None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:34.847838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e126268-187e-4f7b-87c7-b2fef5d26bbc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:34.852034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 136/263] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:34 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:34.852833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e126268-187e-4f7b-87c7-b2fef5d26bbc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:34.898194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:34.915294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852060cd-f01a-45be-ba08-75b7baba6329 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:34.987718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced3148-2086-4363-9188-481a06f8ac33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:34.988321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced3148-2086-4363-9188-481a06f8ac33 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:34.988594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced3148-2086-4363-9188-481a06f8ac33 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:34.988883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced3148-2086-4363-9188-481a06f8ac33 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df/users/be446bcb0abb4e3f96d98ff4b35df27a/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:34.997635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced3148-2086-4363-9188-481a06f8ac33 None admin] RBAC: Authorizing `identity:create_grant(project_id=caedb3d1f7f94504824174418c72d3df, user_id=be446bcb0abb4e3f96d98ff4b35df27a, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:34.999106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced3148-2086-4363-9188-481a06f8ac33 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:35.025895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 128/264] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:42:34 2022] PUT /identity/v3/projects/caedb3d1f7f94504824174418c72d3df/users/be446bcb0abb4e3f96d98ff4b35df27a/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:35.026849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced3148-2086-4363-9188-481a06f8ac33 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:35.072147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e126268-187e-4f7b-87c7-b2fef5d26bbc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:35.200026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced3148-2086-4363-9188-481a06f8ac33 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:35.214199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0f2e3-6329-44b8-9fb2-6f788a9231d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:35.214869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0f2e3-6329-44b8-9fb2-6f788a9231d6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:35.215152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0f2e3-6329-44b8-9fb2-6f788a9231d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:35.215424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0f2e3-6329-44b8-9fb2-6f788a9231d6 None admin] PATH_INFO: `/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:35.216631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0f2e3-6329-44b8-9fb2-6f788a9231d6 None admin] RBAC: Authorizing `identity:get_user(user_id=31e7c1cd7a734d75bfaf8d7604dfadf8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:35.218179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0f2e3-6329-44b8-9fb2-6f788a9231d6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:35.220066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 137/265] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:34 2022] GET /identity/v3/users/31e7c1cd7a734d75bfaf8d7604dfadf8 => generated 283 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:35.229853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0f2e3-6329-44b8-9fb2-6f788a9231d6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:35.244731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ed123-cf66-4002-9300-56fa92fd23b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:35.245313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ed123-cf66-4002-9300-56fa92fd23b6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:35.245592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ed123-cf66-4002-9300-56fa92fd23b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:35.245863 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ed123-cf66-4002-9300-56fa92fd23b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:35.246595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ed123-cf66-4002-9300-56fa92fd23b6 None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=caedb3d1f7f94504824174418c72d3df, user.id=31e7c1cd7a734d75bfaf8d7604dfadf8) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:35.246901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ed123-cf66-4002-9300-56fa92fd23b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:35.248886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ed123-cf66-4002-9300-56fa92fd23b6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:35.256211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 129/266] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:35 2022] GET /identity/v3/role_assignments?user.id=31e7c1cd7a734d75bfaf8d7604dfadf8&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=0be9777002c641b4a47f78df647cc173 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:35.285235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:35.435729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0f2e3-6329-44b8-9fb2-6f788a9231d6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:35.488390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e0f54-eb00-4554-a736-98c8c4275cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:35.493744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e0f54-eb00-4554-a736-98c8c4275cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:35.494075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e0f54-eb00-4554-a736-98c8c4275cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:35.494307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e0f54-eb00-4554-a736-98c8c4275cd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:35.509772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e139b-1f9b-425f-a045-eeac72f51b35 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:35.510256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e139b-1f9b-425f-a045-eeac72f51b35 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:35.510256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e139b-1f9b-425f-a045-eeac72f51b35 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:35.511897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 130/267] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:42:35.520611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17aa253-4ea3-4d9b-8983-ae914ebcf7fb None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:35.520946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17aa253-4ea3-4d9b-8983-ae914ebcf7fb None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:35.521215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17aa253-4ea3-4d9b-8983-ae914ebcf7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:35.550803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-c17aa253-4ea3-4d9b-8983-ae914ebcf7fb None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:35.552103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c17aa253-4ea3-4d9b-8983-ae914ebcf7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:35.573264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:35.610375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e0f54-eb00-4554-a736-98c8c4275cd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:35.612430 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e0f54-eb00-4554-a736-98c8c4275cd0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:35.640637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 131/268] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:35.659897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:35.700237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340d4ff-2022-4369-a224-ca6284b41da0 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:35.700559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340d4ff-2022-4369-a224-ca6284b41da0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:35.700810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340d4ff-2022-4369-a224-ca6284b41da0 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:35.702176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 132/269] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:35.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d8c70-0094-4b6b-846e-723f83e3d47b None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:35.709853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d8c70-0094-4b6b-846e-723f83e3d47b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:35.710114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d8c70-0094-4b6b-846e-723f83e3d47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:35.748834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-8d9d8c70-0094-4b6b-846e-723f83e3d47b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:35.750052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8d9d8c70-0094-4b6b-846e-723f83e3d47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:35.759679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 138/270] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:35.774393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e0f54-eb00-4554-a736-98c8c4275cd0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:35.866236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 133/271] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:35.926709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e0f54-eb00-4554-a736-98c8c4275cd0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:35.950065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:35.969156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb6511-3cda-4879-80b8-4e834180ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:35.969792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb6511-3cda-4879-80b8-4e834180ca29 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:35.970075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb6511-3cda-4879-80b8-4e834180ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:35.970339 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb6511-3cda-4879-80b8-4e834180ca29 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:35.972366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb6511-3cda-4879-80b8-4e834180ca29 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:35.974115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb6511-3cda-4879-80b8-4e834180ca29 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:35.976926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 139/272] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:35 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:35.984644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d8c70-0094-4b6b-846e-723f83e3d47b None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:36.017895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb6511-3cda-4879-80b8-4e834180ca29 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:36.110792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:36.139324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d8c70-0094-4b6b-846e-723f83e3d47b None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:36.183175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf4e3e-32b0-405d-88fa-f12589948cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:36.183785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf4e3e-32b0-405d-88fa-f12589948cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:36.184057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf4e3e-32b0-405d-88fa-f12589948cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:36.184323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf4e3e-32b0-405d-88fa-f12589948cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:36.185078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf4e3e-32b0-405d-88fa-f12589948cd9 None admin] RBAC: Adding query filter params (role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=caedb3d1f7f94504824174418c72d3df, user.id=31e7c1cd7a734d75bfaf8d7604dfadf8) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:36.185381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf4e3e-32b0-405d-88fa-f12589948cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:36.186828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf4e3e-32b0-405d-88fa-f12589948cd9 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:36.207108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 134/273] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:35 2022] GET /identity/v3/role_assignments?user.id=31e7c1cd7a734d75bfaf8d7604dfadf8&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=3518baf98ba54a28a9c79ec287469aff => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:36.243625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb6511-3cda-4879-80b8-4e834180ca29 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:36.271799 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf4e3e-32b0-405d-88fa-f12589948cd9 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:36.335737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f2c64-683d-49ea-aa88-6d145b88e05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:36.336392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f2c64-683d-49ea-aa88-6d145b88e05c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:36.336671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f2c64-683d-49ea-aa88-6d145b88e05c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:36.336968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f2c64-683d-49ea-aa88-6d145b88e05c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:36.367284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f2c64-683d-49ea-aa88-6d145b88e05c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-fd2f2c64-683d-49ea-aa88-6d145b88e05c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:36.375458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:36.376440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:36.377941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 140/274] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:36.385503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f2c64-683d-49ea-aa88-6d145b88e05c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:36.441926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:36.447504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf4e3e-32b0-405d-88fa-f12589948cd9 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:36.494405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a45588-70e0-435d-955b-b34a3baed8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:36.495535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a45588-70e0-435d-955b-b34a3baed8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:36.495964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a45588-70e0-435d-955b-b34a3baed8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:36.496764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a45588-70e0-435d-955b-b34a3baed8a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:36.497942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a45588-70e0-435d-955b-b34a3baed8a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:36.499480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a45588-70e0-435d-955b-b34a3baed8a0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:36.503454 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 135/275] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:36.507973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112c0a2-c1e1-42ce-b0da-9e5bb761413b None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:36.508257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112c0a2-c1e1-42ce-b0da-9e5bb761413b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:36.508532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112c0a2-c1e1-42ce-b0da-9e5bb761413b None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:36.510136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 136/276] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:36.510849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112c0a2-c1e1-42ce-b0da-9e5bb761413b None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:36.516143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f2c64-683d-49ea-aa88-6d145b88e05c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:36.522721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:36.562111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3502462-cc65-4b9e-a29b-c5ad4a16843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:36.562735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3502462-cc65-4b9e-a29b-c5ad4a16843f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:36.563015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3502462-cc65-4b9e-a29b-c5ad4a16843f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:36.563288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3502462-cc65-4b9e-a29b-c5ad4a16843f None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:36.564105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3502462-cc65-4b9e-a29b-c5ad4a16843f None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:36.565144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3502462-cc65-4b9e-a29b-c5ad4a16843f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:36.566569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3502462-cc65-4b9e-a29b-c5ad4a16843f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:36.575858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 141/277] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:36.576962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fb12b-6f38-4c10-b2b7-2bdc8aeeab65 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:36.577239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fb12b-6f38-4c10-b2b7-2bdc8aeeab65 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:36.577514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fb12b-6f38-4c10-b2b7-2bdc8aeeab65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:36.596406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1c4fb12b-6f38-4c10-b2b7-2bdc8aeeab65 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:36.597667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1c4fb12b-6f38-4c10-b2b7-2bdc8aeeab65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:36.635328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112c0a2-c1e1-42ce-b0da-9e5bb761413b None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:36.666847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 142/278] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:36.667629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fb12b-6f38-4c10-b2b7-2bdc8aeeab65 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:36.678253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04ae7d-34e9-4fbc-966f-9f5a261bfb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:36.678949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04ae7d-34e9-4fbc-966f-9f5a261bfb15 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:36.679234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04ae7d-34e9-4fbc-966f-9f5a261bfb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:36.679567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04ae7d-34e9-4fbc-966f-9f5a261bfb15 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:36.680276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04ae7d-34e9-4fbc-966f-9f5a261bfb15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:36.681891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04ae7d-34e9-4fbc-966f-9f5a261bfb15 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:36.716828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 137/279] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:42:36 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:42:36.730545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:36.802443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fb12b-6f38-4c10-b2b7-2bdc8aeeab65 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:36.841435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3a059-0b44-4ad3-9b3d-2f1ed346c2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:36.842088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3a059-0b44-4ad3-9b3d-2f1ed346c2cf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:36.842437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3a059-0b44-4ad3-9b3d-2f1ed346c2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:36.842715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3a059-0b44-4ad3-9b3d-2f1ed346c2cf None admin] PATH_INFO: `/v3/users/8b0aa126a8cc4e2fbb025fc2f7023fc8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:36.858713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3a059-0b44-4ad3-9b3d-2f1ed346c2cf None admin] RBAC: Authorizing `identity:get_user(user_id=8b0aa126a8cc4e2fbb025fc2f7023fc8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:36.860347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3a059-0b44-4ad3-9b3d-2f1ed346c2cf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:36.863161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 143/280] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:36 2022] GET /identity/v3/users/8b0aa126a8cc4e2fbb025fc2f7023fc8 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:36.870169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04ae7d-34e9-4fbc-966f-9f5a261bfb15 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:36.940143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:36.971859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3a059-0b44-4ad3-9b3d-2f1ed346c2cf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:36.989112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04ae7d-34e9-4fbc-966f-9f5a261bfb15 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:37.029961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccef997-1ba3-46e8-a0ac-1ae4e9cce8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:37.030582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccef997-1ba3-46e8-a0ac-1ae4e9cce8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:37.030968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccef997-1ba3-46e8-a0ac-1ae4e9cce8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:37.031249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccef997-1ba3-46e8-a0ac-1ae4e9cce8f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:37.031989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccef997-1ba3-46e8-a0ac-1ae4e9cce8f0 None admin] RBAC: Adding query filter params (role.id=4cef964ca8d143eaadbb479db64250f1, user.id=8b0aa126a8cc4e2fbb025fc2f7023fc8, scope.system=all) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:37.032302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccef997-1ba3-46e8-a0ac-1ae4e9cce8f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:37.034754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccef997-1ba3-46e8-a0ac-1ae4e9cce8f0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:37.042251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 138/281] 10.222.0.50 () {56 vars in 1308 bytes} [Thu Sep 15 10:42:36 2022] GET /identity/v3/role_assignments?user.id=8b0aa126a8cc4e2fbb025fc2f7023fc8&scope.system=all&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:37.082587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:37.090380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3a059-0b44-4ad3-9b3d-2f1ed346c2cf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:37.131052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ea7a0-ba0d-4449-8496-a4b5f97f0b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:37.131721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ea7a0-ba0d-4449-8496-a4b5f97f0b5d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:37.132001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ea7a0-ba0d-4449-8496-a4b5f97f0b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:37.132278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ea7a0-ba0d-4449-8496-a4b5f97f0b5d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:37.145561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ea7a0-ba0d-4449-8496-a4b5f97f0b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-f13ea7a0-ba0d-4449-8496-a4b5f97f0b5d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:37.150734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:37.151641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:37.152459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 144/282] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:37.157972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccef997-1ba3-46e8-a0ac-1ae4e9cce8f0 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:37.299738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccef997-1ba3-46e8-a0ac-1ae4e9cce8f0 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:37.326727 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:37.337276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599e582-fe59-4053-b8b1-5f0822a6bafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:37.337956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599e582-fe59-4053-b8b1-5f0822a6bafa None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:37.338243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599e582-fe59-4053-b8b1-5f0822a6bafa None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:37.338517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599e582-fe59-4053-b8b1-5f0822a6bafa None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:37.339188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599e582-fe59-4053-b8b1-5f0822a6bafa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:37.339495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599e582-fe59-4053-b8b1-5f0822a6bafa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:37.340907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599e582-fe59-4053-b8b1-5f0822a6bafa None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:37.354942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 139/283] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:37.356397 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ea7a0-ba0d-4449-8496-a4b5f97f0b5d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:37.481848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ea7a0-ba0d-4449-8496-a4b5f97f0b5d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:37.518584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddc716-9959-4a9e-a486-09042545a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:37.519404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddc716-9959-4a9e-a486-09042545a932 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:37.519752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddc716-9959-4a9e-a486-09042545a932 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:37.520030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddc716-9959-4a9e-a486-09042545a932 None admin] PATH_INFO: `/v3/users/216176a23a44431f934c25ebac99ecee` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:37.521228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddc716-9959-4a9e-a486-09042545a932 None admin] RBAC: Authorizing `identity:get_user(user_id=216176a23a44431f934c25ebac99ecee)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:37.522736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddc716-9959-4a9e-a486-09042545a932 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:37.524510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 145/284] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:37 2022] GET /identity/v3/users/216176a23a44431f934c25ebac99ecee => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:37.530160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599e582-fe59-4053-b8b1-5f0822a6bafa None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:37.676935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599e582-fe59-4053-b8b1-5f0822a6bafa None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:37.722364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4d587-15b0-4517-ba1c-a400f0d670b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:37.723011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4d587-15b0-4517-ba1c-a400f0d670b6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:37.723289 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4d587-15b0-4517-ba1c-a400f0d670b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:37.723561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4d587-15b0-4517-ba1c-a400f0d670b6 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:37.724707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4d587-15b0-4517-ba1c-a400f0d670b6 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:37.726189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4d587-15b0-4517-ba1c-a400f0d670b6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:37.728115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 140/285] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:37 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:37.733675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddc716-9959-4a9e-a486-09042545a932 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:37.780705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:37.889328 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddc716-9959-4a9e-a486-09042545a932 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:37.909178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fd5df-5624-4605-930d-5a32aab769b6 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:37.910177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fd5df-5624-4605-930d-5a32aab769b6 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:37.910650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fd5df-5624-4605-930d-5a32aab769b6 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:37.912306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 141/286] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:42:37.928141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b2b9d-8860-49f0-868a-4a1c5f594056 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:37.929146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b2b9d-8860-49f0-868a-4a1c5f594056 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:37.929916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b2b9d-8860-49f0-868a-4a1c5f594056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:37.947857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1183d-fb85-4e0b-83aa-acfa0e0d747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:37.948595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1183d-fb85-4e0b-83aa-acfa0e0d747e None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:37.948922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1183d-fb85-4e0b-83aa-acfa0e0d747e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:37.949203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1183d-fb85-4e0b-83aa-acfa0e0d747e None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df/users/216176a23a44431f934c25ebac99ecee/roles/fdf0d588fb134183911419bbaa627d22` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:37.951147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1183d-fb85-4e0b-83aa-acfa0e0d747e None admin] RBAC: Authorizing `identity:create_grant(project_id=caedb3d1f7f94504824174418c72d3df, user_id=216176a23a44431f934c25ebac99ecee, role_id=fdf0d588fb134183911419bbaa627d22)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:37.952616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1183d-fb85-4e0b-83aa-acfa0e0d747e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:37.964338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4c9b2b9d-8860-49f0-868a-4a1c5f594056 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:37.965641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4c9b2b9d-8860-49f0-868a-4a1c5f594056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:37.979697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 146/287] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:42:37 2022] PUT /identity/v3/projects/caedb3d1f7f94504824174418c72d3df/users/216176a23a44431f934c25ebac99ecee/roles/fdf0d588fb134183911419bbaa627d22 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:38.039762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:38.061340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 142/288] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:38.369749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1183d-fb85-4e0b-83aa-acfa0e0d747e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:38.486616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:38.509913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1183d-fb85-4e0b-83aa-acfa0e0d747e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:38.568958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17abc05-3f51-4c7b-9114-751d577c7483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:38.569607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17abc05-3f51-4c7b-9114-751d577c7483 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:38.569890 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17abc05-3f51-4c7b-9114-751d577c7483 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:38.570165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17abc05-3f51-4c7b-9114-751d577c7483 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:38.588594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17abc05-3f51-4c7b-9114-751d577c7483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-d17abc05-3f51-4c7b-9114-751d577c7483 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:38.593597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:38.594487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:38.595300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 147/289] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:38.600995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b2b9d-8860-49f0-868a-4a1c5f594056 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:38.731075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b2b9d-8860-49f0-868a-4a1c5f594056 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:38.786818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0cda4-3571-4941-847f-06887050354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:38.787849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0cda4-3571-4941-847f-06887050354c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:38.788330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0cda4-3571-4941-847f-06887050354c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:38.788791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0cda4-3571-4941-847f-06887050354c None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:38.789685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0cda4-3571-4941-847f-06887050354c None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:38.790194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0cda4-3571-4941-847f-06887050354c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:38.791754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0cda4-3571-4941-847f-06887050354c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:38.803913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 143/290] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:38.809827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17abc05-3f51-4c7b-9114-751d577c7483 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:38.973569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17abc05-3f51-4c7b-9114-751d577c7483 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:39.028993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bb51a-2f83-4852-a20a-a45d8eff3cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:39.029559 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bb51a-2f83-4852-a20a-a45d8eff3cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:39.029842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bb51a-2f83-4852-a20a-a45d8eff3cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:39.030119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bb51a-2f83-4852-a20a-a45d8eff3cc5 None admin] PATH_INFO: `/v3/users/be446bcb0abb4e3f96d98ff4b35df27a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:39.031261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bb51a-2f83-4852-a20a-a45d8eff3cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=be446bcb0abb4e3f96d98ff4b35df27a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:39.032833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bb51a-2f83-4852-a20a-a45d8eff3cc5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:39.035383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 148/291] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:38 2022] GET /identity/v3/users/be446bcb0abb4e3f96d98ff4b35df27a => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:39.040971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0cda4-3571-4941-847f-06887050354c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:39.138586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:39.207215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0cda4-3571-4941-847f-06887050354c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:39.257002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd511aca-39ca-4cd5-a54e-8cecaf641d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:39.257621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511aca-39ca-4cd5-a54e-8cecaf641d0f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:39.257947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511aca-39ca-4cd5-a54e-8cecaf641d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:39.258226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511aca-39ca-4cd5-a54e-8cecaf641d0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:39.341698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd511aca-39ca-4cd5-a54e-8cecaf641d0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:39.343081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd511aca-39ca-4cd5-a54e-8cecaf641d0f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:39.410147 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:39.473973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 144/292] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:39.479186 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bb51a-2f83-4852-a20a-a45d8eff3cc5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:39.503542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:39.636925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bb51a-2f83-4852-a20a-a45d8eff3cc5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:39.684674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885084d-7535-4463-b4e1-6a89079c1f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:39.685345 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885084d-7535-4463-b4e1-6a89079c1f82 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:39.685628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885084d-7535-4463-b4e1-6a89079c1f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:39.685902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885084d-7535-4463-b4e1-6a89079c1f82 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:39.687027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885084d-7535-4463-b4e1-6a89079c1f82 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:39.688509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885084d-7535-4463-b4e1-6a89079c1f82 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:39.690962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 149/293] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:39 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:39.696740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd511aca-39ca-4cd5-a54e-8cecaf641d0f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:39.852176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd511aca-39ca-4cd5-a54e-8cecaf641d0f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:39.901373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f58d9-b5fb-4baf-81a2-66ea8bd91464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:39.901990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f58d9-b5fb-4baf-81a2-66ea8bd91464 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:39.902266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f58d9-b5fb-4baf-81a2-66ea8bd91464 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:39.902535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f58d9-b5fb-4baf-81a2-66ea8bd91464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:39.903261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f58d9-b5fb-4baf-81a2-66ea8bd91464 None admin] RBAC: Adding query filter params (role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=caedb3d1f7f94504824174418c72d3df, user.id=be446bcb0abb4e3f96d98ff4b35df27a) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:39.903565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f58d9-b5fb-4baf-81a2-66ea8bd91464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:39.905028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f58d9-b5fb-4baf-81a2-66ea8bd91464 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:39.916592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 145/294] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:39 2022] GET /identity/v3/role_assignments?user.id=be446bcb0abb4e3f96d98ff4b35df27a&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:39.919579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cce45c-9143-4310-984c-88b360e32e46 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:39.920054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cce45c-9143-4310-984c-88b360e32e46 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:39.920143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cce45c-9143-4310-984c-88b360e32e46 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:39.921484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 150/295] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:39.937823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5f96c-5c3a-4c5d-8b9f-fa94f8b78364 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:39.938127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5f96c-5c3a-4c5d-8b9f-fa94f8b78364 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:39.938380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5f96c-5c3a-4c5d-8b9f-fa94f8b78364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:39.964995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dca4c-be36-4144-9b57-8b987d21108e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:39.965430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-f8b5f96c-5c3a-4c5d-8b9f-fa94f8b78364 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:39.965830 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dca4c-be36-4144-9b57-8b987d21108e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:39.966305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dca4c-be36-4144-9b57-8b987d21108e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:39.966705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f8b5f96c-5c3a-4c5d-8b9f-fa94f8b78364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:39.968492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 151/296] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:39.974940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacdd37-ec4d-4cc3-972f-af5e38d3b6ca None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:39.975532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacdd37-ec4d-4cc3-972f-af5e38d3b6ca None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:39.976013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacdd37-ec4d-4cc3-972f-af5e38d3b6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:39.977077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:39.999730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7eacdd37-ec4d-4cc3-972f-af5e38d3b6ca None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:40.001346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7eacdd37-ec4d-4cc3-972f-af5e38d3b6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:40.067880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 146/297] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:40.069595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd29d06-aa89-4ee3-b6bb-5c6f9a393630 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.070163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd29d06-aa89-4ee3-b6bb-5c6f9a393630 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.070654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd29d06-aa89-4ee3-b6bb-5c6f9a393630 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.073218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 147/298] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:40.081596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ab6c9-39d2-40c7-81a0-e93c46c7956f None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.081942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ab6c9-39d2-40c7-81a0-e93c46c7956f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.082223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ab6c9-39d2-40c7-81a0-e93c46c7956f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.097506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 152/299] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:40.108206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-107ab6c9-39d2-40c7-81a0-e93c46c7956f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:40.108206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-107ab6c9-39d2-40c7-81a0-e93c46c7956f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:40.182481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 148/300] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:40.393348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacdd37-ec4d-4cc3-972f-af5e38d3b6ca None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:40.406203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:40.412491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ab6c9-39d2-40c7-81a0-e93c46c7956f None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:40.509272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacdd37-ec4d-4cc3-972f-af5e38d3b6ca None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:40.515452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:40.549600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e79b-4ff5-4722-89d5-8885c680bf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:40.550367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e79b-4ff5-4722-89d5-8885c680bf08 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.550649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e79b-4ff5-4722-89d5-8885c680bf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.550926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e79b-4ff5-4722-89d5-8885c680bf08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.552143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77e79b-4ff5-4722-89d5-8885c680bf08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:40.553666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77e79b-4ff5-4722-89d5-8885c680bf08 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:40.555315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 153/301] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:40.559951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ab6c9-39d2-40c7-81a0-e93c46c7956f None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:40.561585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e79b-4ff5-4722-89d5-8885c680bf08 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:40.601115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b2bc-eac3-4fc8-9cd9-9dd9e5292e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:40.601762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b2bc-eac3-4fc8-9cd9-9dd9e5292e96 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.602044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b2bc-eac3-4fc8-9cd9-9dd9e5292e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.602315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b2bc-eac3-4fc8-9cd9-9dd9e5292e96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.614419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9b2bc-eac3-4fc8-9cd9-9dd9e5292e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-39e9b2bc-eac3-4fc8-9cd9-9dd9e5292e96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:40.619579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:40.620548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:40.621406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:40.622023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 149/302] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:40.622641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b2bc-eac3-4fc8-9cd9-9dd9e5292e96 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:40.649185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:40.670400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e79b-4ff5-4722-89d5-8885c680bf08 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:40.703707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9498500-23d8-4713-a3f9-fdf7d6970579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:40.704333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9498500-23d8-4713-a3f9-fdf7d6970579 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.704615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9498500-23d8-4713-a3f9-fdf7d6970579 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.705487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9498500-23d8-4713-a3f9-fdf7d6970579 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.715960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9498500-23d8-4713-a3f9-fdf7d6970579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e9498500-23d8-4713-a3f9-fdf7d6970579 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:40.721044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:40.722131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 154/303] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:40.723114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9498500-23d8-4713-a3f9-fdf7d6970579 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:40.726972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b2bc-eac3-4fc8-9cd9-9dd9e5292e96 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:40.757487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:40.763973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02a7e0-0c2d-49e5-9602-45a94d31f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:40.764870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02a7e0-0c2d-49e5-9602-45a94d31f934 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.765155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02a7e0-0c2d-49e5-9602-45a94d31f934 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.765713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02a7e0-0c2d-49e5-9602-45a94d31f934 None admin] PATH_INFO: `/v3/groups` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.766054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02a7e0-0c2d-49e5-9602-45a94d31f934 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:40.768104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02a7e0-0c2d-49e5-9602-45a94d31f934 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:40.783685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 150/304] 10.222.0.50 () {60 vars in 1134 bytes} [Thu Sep 15 10:42:40 2022] POST /identity/v3/groups => generated 229 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:42:40.785283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259e16b-96c5-4a58-8f4c-a99c3ae08496 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.785567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259e16b-96c5-4a58-8f4c-a99c3ae08496 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.785839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259e16b-96c5-4a58-8f4c-a99c3ae08496 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.787086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 151/305] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:40.787802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259e16b-96c5-4a58-8f4c-a99c3ae08496 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:40.854852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9498500-23d8-4713-a3f9-fdf7d6970579 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:40.893718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccad3e-2863-4e3f-8757-ee209d0d418b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:40.894335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccad3e-2863-4e3f-8757-ee209d0d418b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.894612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccad3e-2863-4e3f-8757-ee209d0d418b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.894886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccad3e-2863-4e3f-8757-ee209d0d418b None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.895560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccad3e-2863-4e3f-8757-ee209d0d418b None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:40.895924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccad3e-2863-4e3f-8757-ee209d0d418b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:40.897357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccad3e-2863-4e3f-8757-ee209d0d418b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:40.903096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:40.910432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259e16b-96c5-4a58-8f4c-a99c3ae08496 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:40.912402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 155/306] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:40.913953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7240636-22a5-4565-b675-62ba19f93390 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.914243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7240636-22a5-4565-b675-62ba19f93390 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.914516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7240636-22a5-4565-b675-62ba19f93390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.933742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a7240636-22a5-4565-b675-62ba19f93390 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:40.934961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a7240636-22a5-4565-b675-62ba19f93390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:40.949382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ffba9-23b7-4602-ab6c-0b3d791d80cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:40.950006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ffba9-23b7-4602-ab6c-0b3d791d80cf None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:40.950290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ffba9-23b7-4602-ab6c-0b3d791d80cf None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:40.950568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ffba9-23b7-4602-ab6c-0b3d791d80cf None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:40.951234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ffba9-23b7-4602-ab6c-0b3d791d80cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:40.951539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ffba9-23b7-4602-ab6c-0b3d791d80cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:40.953249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ffba9-23b7-4602-ab6c-0b3d791d80cf None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:40.967914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 152/307] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:40.967914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ffba9-23b7-4602-ab6c-0b3d791d80cf None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:41.015461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 156/308] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:41.016232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7240636-22a5-4565-b675-62ba19f93390 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:41.029664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:41.082959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ffba9-23b7-4602-ab6c-0b3d791d80cf None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:41.122674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412cf46-fbbf-42dc-a21c-862076d49cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:41.123586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412cf46-fbbf-42dc-a21c-862076d49cff None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:41.124013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412cf46-fbbf-42dc-a21c-862076d49cff None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:41.124429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412cf46-fbbf-42dc-a21c-862076d49cff None admin] PATH_INFO: `/v3/users/8b0aa126a8cc4e2fbb025fc2f7023fc8` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:41.125889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412cf46-fbbf-42dc-a21c-862076d49cff None admin] RBAC: Authorizing `identity:get_user(user_id=8b0aa126a8cc4e2fbb025fc2f7023fc8)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:41.128108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412cf46-fbbf-42dc-a21c-862076d49cff None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:41.130182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 153/309] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:40 2022] GET /identity/v3/users/8b0aa126a8cc4e2fbb025fc2f7023fc8 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:41.135377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412cf46-fbbf-42dc-a21c-862076d49cff None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:41.146538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7240636-22a5-4565-b675-62ba19f93390 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:41.179184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:41.185407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211fe4b-c7f1-4556-9a0d-6681f512ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:41.186261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211fe4b-c7f1-4556-9a0d-6681f512ea20 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:41.186558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211fe4b-c7f1-4556-9a0d-6681f512ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:41.187177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211fe4b-c7f1-4556-9a0d-6681f512ea20 None admin] PATH_INFO: `/v3/users/a48b886f49bb4111919a535126b3a7b2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:41.202316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211fe4b-c7f1-4556-9a0d-6681f512ea20 None admin] RBAC: Authorizing `identity:get_user(user_id=a48b886f49bb4111919a535126b3a7b2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:41.204460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211fe4b-c7f1-4556-9a0d-6681f512ea20 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:41.207069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 157/310] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:41 2022] GET /identity/v3/users/a48b886f49bb4111919a535126b3a7b2 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:41.212387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211fe4b-c7f1-4556-9a0d-6681f512ea20 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:41.250673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412cf46-fbbf-42dc-a21c-862076d49cff None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:41.275564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:41.290314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddfcf3-8332-47d6-b81d-d1c1b21122b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:41.291284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddfcf3-8332-47d6-b81d-d1c1b21122b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:41.291807 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddfcf3-8332-47d6-b81d-d1c1b21122b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:41.292284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddfcf3-8332-47d6-b81d-d1c1b21122b0 None admin] PATH_INFO: `/v3/system/users/8b0aa126a8cc4e2fbb025fc2f7023fc8/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:41.293971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddfcf3-8332-47d6-b81d-d1c1b21122b0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8b0aa126a8cc4e2fbb025fc2f7023fc8, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:41.295603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddfcf3-8332-47d6-b81d-d1c1b21122b0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:41.307124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 154/311] 10.222.0.50 () {58 vars in 1334 bytes} [Thu Sep 15 10:42:41 2022] PUT /identity/v3/system/users/8b0aa126a8cc4e2fbb025fc2f7023fc8/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:42:41.308089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddfcf3-8332-47d6-b81d-d1c1b21122b0 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:41.341502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211fe4b-c7f1-4556-9a0d-6681f512ea20 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:41.387779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68025888-6b4e-4515-a50d-21a01a7d404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:41.388551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68025888-6b4e-4515-a50d-21a01a7d404d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:41.389100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68025888-6b4e-4515-a50d-21a01a7d404d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:41.389445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68025888-6b4e-4515-a50d-21a01a7d404d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:41.390703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68025888-6b4e-4515-a50d-21a01a7d404d None admin] RBAC: Adding query filter params (role.id=fdf0d588fb134183911419bbaa627d22, user.id=a48b886f49bb4111919a535126b3a7b2, scope.system=all) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:41.391163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68025888-6b4e-4515-a50d-21a01a7d404d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:41.393178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68025888-6b4e-4515-a50d-21a01a7d404d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:41.405296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 158/312] 10.222.0.50 () {56 vars in 1308 bytes} [Thu Sep 15 10:42:41 2022] GET /identity/v3/role_assignments?user.id=a48b886f49bb4111919a535126b3a7b2&scope.system=all&role.id=fdf0d588fb134183911419bbaa627d22 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:41.442043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddfcf3-8332-47d6-b81d-d1c1b21122b0 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:41.483090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f379e0-1c52-45bf-a7f7-c8ee24b19c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:41.484161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f379e0-1c52-45bf-a7f7-c8ee24b19c1b None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:41.484635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f379e0-1c52-45bf-a7f7-c8ee24b19c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:41.485154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f379e0-1c52-45bf-a7f7-c8ee24b19c1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:41.498988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f379e0-1c52-45bf-a7f7-c8ee24b19c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-23f379e0-1c52-45bf-a7f7-c8ee24b19c1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:41.505526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:41.506521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:41.507357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:41.507976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 155/313] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:41.514440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68025888-6b4e-4515-a50d-21a01a7d404d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:41.577330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:41.629823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68025888-6b4e-4515-a50d-21a01a7d404d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:41.668654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f11ff3-8b4d-4e69-a504-b7b6a90da138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:41.669286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f11ff3-8b4d-4e69-a504-b7b6a90da138 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:41.669563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f11ff3-8b4d-4e69-a504-b7b6a90da138 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:41.669834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f11ff3-8b4d-4e69-a504-b7b6a90da138 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:41.670490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f11ff3-8b4d-4e69-a504-b7b6a90da138 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:41.670789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f11ff3-8b4d-4e69-a504-b7b6a90da138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:41.672156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f11ff3-8b4d-4e69-a504-b7b6a90da138 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:41.681789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 159/314] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:41.687325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f379e0-1c52-45bf-a7f7-c8ee24b19c1b None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:41.746169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:41.806658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f379e0-1c52-45bf-a7f7-c8ee24b19c1b None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:41.841848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e711cb-9fcd-4b42-ba6d-2637cab8377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:41.842478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e711cb-9fcd-4b42-ba6d-2637cab8377d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:41.842756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e711cb-9fcd-4b42-ba6d-2637cab8377d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:41.843030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e711cb-9fcd-4b42-ba6d-2637cab8377d None admin] PATH_INFO: `/v3/users/216176a23a44431f934c25ebac99ecee` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:41.844159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e711cb-9fcd-4b42-ba6d-2637cab8377d None admin] RBAC: Authorizing `identity:get_user(user_id=216176a23a44431f934c25ebac99ecee)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:41.846620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e711cb-9fcd-4b42-ba6d-2637cab8377d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:41.848607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 156/315] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:41 2022] GET /identity/v3/users/216176a23a44431f934c25ebac99ecee => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:41.854102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f11ff3-8b4d-4e69-a504-b7b6a90da138 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:41.962758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f11ff3-8b4d-4e69-a504-b7b6a90da138 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:41.977406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:41.999238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a3dfa-3abf-4394-ba37-869ac1a61425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:41.999862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a3dfa-3abf-4394-ba37-869ac1a61425 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:42.000152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a3dfa-3abf-4394-ba37-869ac1a61425 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:42.000428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a3dfa-3abf-4394-ba37-869ac1a61425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:42.076886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a3dfa-3abf-4394-ba37-869ac1a61425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:42.078554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a3dfa-3abf-4394-ba37-869ac1a61425 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:42.186595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 160/316] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:42.193505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e711cb-9fcd-4b42-ba6d-2637cab8377d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:42.305943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e711cb-9fcd-4b42-ba6d-2637cab8377d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:42.320725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:42.341808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441441f-129f-4cae-884a-caf79aac4112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:42.342429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441441f-129f-4cae-884a-caf79aac4112 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:42.342712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441441f-129f-4cae-884a-caf79aac4112 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:42.342985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441441f-129f-4cae-884a-caf79aac4112 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:42.344106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441441f-129f-4cae-884a-caf79aac4112 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:42.345807 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441441f-129f-4cae-884a-caf79aac4112 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:42.347857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 157/317] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:42 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:42.357985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a3dfa-3abf-4394-ba37-869ac1a61425 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:42.374891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:42.466458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a3dfa-3abf-4394-ba37-869ac1a61425 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:42.502935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196835c-62b9-4e30-bb22-56361e2306e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:42.503862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196835c-62b9-4e30-bb22-56361e2306e8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:42.504318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196835c-62b9-4e30-bb22-56361e2306e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:42.504795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196835c-62b9-4e30-bb22-56361e2306e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:42.505712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196835c-62b9-4e30-bb22-56361e2306e8 None admin] RBAC: Adding query filter params (role.id=fdf0d588fb134183911419bbaa627d22, scope.project.id=caedb3d1f7f94504824174418c72d3df, user.id=216176a23a44431f934c25ebac99ecee) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:42.506196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196835c-62b9-4e30-bb22-56361e2306e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:42.507809 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196835c-62b9-4e30-bb22-56361e2306e8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:42.514386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 161/318] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:42:42 2022] GET /identity/v3/role_assignments?user.id=216176a23a44431f934c25ebac99ecee&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=fdf0d588fb134183911419bbaa627d22 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:43.059753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57726eb1-4d04-43f4-b705-abbfb6ebb42c None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:43.060269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57726eb1-4d04-43f4-b705-abbfb6ebb42c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:43.060345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57726eb1-4d04-43f4-b705-abbfb6ebb42c None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:43.061800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 158/319] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:43.067865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfaa3f-41bf-46d1-90eb-b11d1bf839b6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:43.068267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfaa3f-41bf-46d1-90eb-b11d1bf839b6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:43.068428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfaa3f-41bf-46d1-90eb-b11d1bf839b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:43.087277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bdcfaa3f-41bf-46d1-90eb-b11d1bf839b6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:43.088449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bdcfaa3f-41bf-46d1-90eb-b11d1bf839b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:43.158985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 162/320] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:43.430344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57726eb1-4d04-43f4-b705-abbfb6ebb42c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:43.539841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57726eb1-4d04-43f4-b705-abbfb6ebb42c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:43.579508 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f132757-8599-41d6-930d-a0c3e7fd9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:43.580771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f132757-8599-41d6-930d-a0c3e7fd9298 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:43.581083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f132757-8599-41d6-930d-a0c3e7fd9298 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:43.581352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f132757-8599-41d6-930d-a0c3e7fd9298 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:43.582510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f132757-8599-41d6-930d-a0c3e7fd9298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:43.583993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f132757-8599-41d6-930d-a0c3e7fd9298 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:43.588847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 159/321] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:42:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:43.591263 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfaa3f-41bf-46d1-90eb-b11d1bf839b6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:43.598832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6313e-c13b-4951-a2d5-f0225afbb981 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:43.599113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6313e-c13b-4951-a2d5-f0225afbb981 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:43.599418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6313e-c13b-4951-a2d5-f0225afbb981 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:43.600844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 160/322] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:42:43.601665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:43.605867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aaa57-4f86-4967-a635-3d7ddc53e388 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:43.606148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aaa57-4f86-4967-a635-3d7ddc53e388 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:43.606418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aaa57-4f86-4967-a635-3d7ddc53e388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:43.625323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-561aaa57-4f86-4967-a635-3d7ddc53e388 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:43.627089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-561aaa57-4f86-4967-a635-3d7ddc53e388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:43.676356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:43.696514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfaa3f-41bf-46d1-90eb-b11d1bf839b6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:43.697207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 161/323] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:43.698434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd61107-6736-4f5b-ba3c-f8efa23c255a None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:43.698721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd61107-6736-4f5b-ba3c-f8efa23c255a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:43.698995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd61107-6736-4f5b-ba3c-f8efa23c255a None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:43.700397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 162/324] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:43.705687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564791b-66a4-40e9-91d2-086e80e42e07 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:43.705969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564791b-66a4-40e9-91d2-086e80e42e07 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:43.706241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564791b-66a4-40e9-91d2-086e80e42e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:43.724762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7564791b-66a4-40e9-91d2-086e80e42e07 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:43.725995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7564791b-66a4-40e9-91d2-086e80e42e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:43.732714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052123b-7cf3-4886-89ad-4868b9a9144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:43.733763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052123b-7cf3-4886-89ad-4868b9a9144c None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:43.734051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052123b-7cf3-4886-89ad-4868b9a9144c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:43.734323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052123b-7cf3-4886-89ad-4868b9a9144c None admin] PATH_INFO: `/v3/groups` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:43.734966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052123b-7cf3-4886-89ad-4868b9a9144c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:43.738004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052123b-7cf3-4886-89ad-4868b9a9144c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:43.757150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 163/325] 10.222.0.50 () {60 vars in 1134 bytes} [Thu Sep 15 10:42:43 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:42:43.812077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 163/326] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:43.967796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052123b-7cf3-4886-89ad-4868b9a9144c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:44.041497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:44.081567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052123b-7cf3-4886-89ad-4868b9a9144c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:44.088302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564791b-66a4-40e9-91d2-086e80e42e07 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:44.119216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527fc47-6374-491b-a808-3032d06b91f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:44.119851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527fc47-6374-491b-a808-3032d06b91f8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:44.120127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527fc47-6374-491b-a808-3032d06b91f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:44.120392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527fc47-6374-491b-a808-3032d06b91f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:44.132231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527fc47-6374-491b-a808-3032d06b91f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2527fc47-6374-491b-a808-3032d06b91f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:44.138054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:44.138992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:44.139816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 164/327] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:44.145242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527fc47-6374-491b-a808-3032d06b91f8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:44.219885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564791b-66a4-40e9-91d2-086e80e42e07 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:44.231638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:44.262790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7124d7-db35-42ad-86df-3828a0ca161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:44.263436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7124d7-db35-42ad-86df-3828a0ca161f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:44.263716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7124d7-db35-42ad-86df-3828a0ca161f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:44.263991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7124d7-db35-42ad-86df-3828a0ca161f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:44.273025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527fc47-6374-491b-a808-3032d06b91f8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:44.278698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7124d7-db35-42ad-86df-3828a0ca161f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-9e7124d7-db35-42ad-86df-3828a0ca161f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:44.284531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:44.285667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:44.286775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 164/328] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:44.294969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7124d7-db35-42ad-86df-3828a0ca161f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:44.310128 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:44.321598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dac8a-a3eb-410e-bf24-05f428f4de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:44.322435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dac8a-a3eb-410e-bf24-05f428f4de20 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:44.322860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dac8a-a3eb-410e-bf24-05f428f4de20 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:44.323304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dac8a-a3eb-410e-bf24-05f428f4de20 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:44.324655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70dac8a-a3eb-410e-bf24-05f428f4de20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:44.325284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70dac8a-a3eb-410e-bf24-05f428f4de20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:44.327248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70dac8a-a3eb-410e-bf24-05f428f4de20 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:44.344286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 165/329] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:44.348177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dac8a-a3eb-410e-bf24-05f428f4de20 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:44.425587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7124d7-db35-42ad-86df-3828a0ca161f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:44.464754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5d5a9-f971-45bc-b369-e52c7585ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:44.465507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5d5a9-f971-45bc-b369-e52c7585ac9b None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:44.465906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5d5a9-f971-45bc-b369-e52c7585ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:44.466179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5d5a9-f971-45bc-b369-e52c7585ac9b None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:44.466848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5d5a9-f971-45bc-b369-e52c7585ac9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:44.467155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5d5a9-f971-45bc-b369-e52c7585ac9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:44.468628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5d5a9-f971-45bc-b369-e52c7585ac9b None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:44.473112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dac8a-a3eb-410e-bf24-05f428f4de20 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:44.479720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 165/330] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:44.487264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5d5a9-f971-45bc-b369-e52c7585ac9b None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:44.511591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af744-ada4-4156-9a95-a84915255058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:44.512231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af744-ada4-4156-9a95-a84915255058 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:44.512514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af744-ada4-4156-9a95-a84915255058 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:44.512787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af744-ada4-4156-9a95-a84915255058 None admin] PATH_INFO: `/v3/users/8b0aa126a8cc4e2fbb025fc2f7023fc8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:44.513966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384af744-ada4-4156-9a95-a84915255058 None admin] RBAC: Authorizing `identity:get_user(user_id=8b0aa126a8cc4e2fbb025fc2f7023fc8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:44.515440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384af744-ada4-4156-9a95-a84915255058 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:44.518228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 166/331] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:44 2022] GET /identity/v3/users/8b0aa126a8cc4e2fbb025fc2f7023fc8 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:44.527858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:44.528492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af744-ada4-4156-9a95-a84915255058 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:44.630098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:44.647251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5d5a9-f971-45bc-b369-e52c7585ac9b None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:44.689955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d7d92-f7b3-406b-b100-97a98d3b514f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:44.690945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d7d92-f7b3-406b-b100-97a98d3b514f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:44.691070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af744-ada4-4156-9a95-a84915255058 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:44.691545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d7d92-f7b3-406b-b100-97a98d3b514f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:44.692071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d7d92-f7b3-406b-b100-97a98d3b514f None admin] PATH_INFO: `/v3/users/a48b886f49bb4111919a535126b3a7b2` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:44.693699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d7d92-f7b3-406b-b100-97a98d3b514f None admin] RBAC: Authorizing `identity:get_user(user_id=a48b886f49bb4111919a535126b3a7b2)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:44.695343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d7d92-f7b3-406b-b100-97a98d3b514f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:44.697165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 166/332] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:44 2022] GET /identity/v3/users/a48b886f49bb4111919a535126b3a7b2 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:44.703086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d7d92-f7b3-406b-b100-97a98d3b514f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:44.747735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1fa06-4530-43b7-8cc6-8caa5dcfffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:44.748330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1fa06-4530-43b7-8cc6-8caa5dcfffd7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:44.748615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1fa06-4530-43b7-8cc6-8caa5dcfffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:44.748926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1fa06-4530-43b7-8cc6-8caa5dcfffd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:44.749658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1fa06-4530-43b7-8cc6-8caa5dcfffd7 None admin] RBAC: Adding query filter params (role.id=4cef964ca8d143eaadbb479db64250f1, user.id=8b0aa126a8cc4e2fbb025fc2f7023fc8, scope.system=all) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:44.749967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1fa06-4530-43b7-8cc6-8caa5dcfffd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:44.752105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1fa06-4530-43b7-8cc6-8caa5dcfffd7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:44.766343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 167/333] 10.222.0.50 () {56 vars in 1308 bytes} [Thu Sep 15 10:42:44 2022] GET /identity/v3/role_assignments?user.id=8b0aa126a8cc4e2fbb025fc2f7023fc8&scope.system=all&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 510 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:44.823363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:44.826708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d7d92-f7b3-406b-b100-97a98d3b514f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:44.876233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea568cbb-c254-4a31-a730-099e1988ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:44.877794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea568cbb-c254-4a31-a730-099e1988ba6e None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:44.877903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea568cbb-c254-4a31-a730-099e1988ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:44.878104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea568cbb-c254-4a31-a730-099e1988ba6e None admin] PATH_INFO: `/v3/system/users/a48b886f49bb4111919a535126b3a7b2/roles/fdf0d588fb134183911419bbaa627d22` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:44.885818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea568cbb-c254-4a31-a730-099e1988ba6e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a48b886f49bb4111919a535126b3a7b2, role_id=fdf0d588fb134183911419bbaa627d22)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:44.887268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea568cbb-c254-4a31-a730-099e1988ba6e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:44.902512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 167/334] 10.222.0.50 () {58 vars in 1334 bytes} [Thu Sep 15 10:42:44 2022] PUT /identity/v3/system/users/a48b886f49bb4111919a535126b3a7b2/roles/fdf0d588fb134183911419bbaa627d22 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:42:46.573284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839882-0e0f-46b4-b1fc-baca66a8f378 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:46.573826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839882-0e0f-46b4-b1fc-baca66a8f378 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:46.573826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839882-0e0f-46b4-b1fc-baca66a8f378 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:46.575225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 168/335] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:46.582325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8abecc-a5fc-402d-bd47-28c026e75089 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:46.582840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8abecc-a5fc-402d-bd47-28c026e75089 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:46.583099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8abecc-a5fc-402d-bd47-28c026e75089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:46.603602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-8f8abecc-a5fc-402d-bd47-28c026e75089 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:46.604830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8f8abecc-a5fc-402d-bd47-28c026e75089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:46.695984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3954d-f13e-41f0-99e1-a9fbee4270c6 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:46.696300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3954d-f13e-41f0-99e1-a9fbee4270c6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:46.696544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3954d-f13e-41f0-99e1-a9fbee4270c6 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:46.698104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 169/336] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:46.699032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 168/337] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:46.709344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25867827-3ece-4344-aec2-572fb67c1686 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:46.709636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25867827-3ece-4344-aec2-572fb67c1686 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:46.709912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25867827-3ece-4344-aec2-572fb67c1686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:46.737820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-25867827-3ece-4344-aec2-572fb67c1686 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:46.739047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-25867827-3ece-4344-aec2-572fb67c1686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:46.825220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 170/338] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:47.221292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8abecc-a5fc-402d-bd47-28c026e75089 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:47.271502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25867827-3ece-4344-aec2-572fb67c1686 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:47.377177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:47.381798 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:47.387061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8abecc-a5fc-402d-bd47-28c026e75089 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:47.442911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdf711-06b8-42a0-831d-940d91db8791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:47.443564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdf711-06b8-42a0-831d-940d91db8791 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:47.443844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdf711-06b8-42a0-831d-940d91db8791 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:47.444118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdf711-06b8-42a0-831d-940d91db8791 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:47.466265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdf711-06b8-42a0-831d-940d91db8791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-81bdf711-06b8-42a0-831d-940d91db8791 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:47.471885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:47.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:47.473676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:47.477260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 169/339] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:47.482836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdf711-06b8-42a0-831d-940d91db8791 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:47.489270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25867827-3ece-4344-aec2-572fb67c1686 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:47.594071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bf273-44a7-4363-9916-29bf19fd2c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:47.596699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bf273-44a7-4363-9916-29bf19fd2c22 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:47.597140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bf273-44a7-4363-9916-29bf19fd2c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:47.597532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bf273-44a7-4363-9916-29bf19fd2c22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:47.611759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bf273-44a7-4363-9916-29bf19fd2c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-f24bf273-44a7-4363-9916-29bf19fd2c22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:47.616789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:47.617688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:47.618914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 171/340] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:47.619804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e6d5-d36a-432f-ae92-40fe0e723d6d None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:47.620832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e6d5-d36a-432f-ae92-40fe0e723d6d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:47.620832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e6d5-d36a-432f-ae92-40fe0e723d6d None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:47.625817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 172/341] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:47.631530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98e924-36a2-44d8-9f0a-f7404c56c2b1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:47.631817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98e924-36a2-44d8-9f0a-f7404c56c2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:47.632188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98e924-36a2-44d8-9f0a-f7404c56c2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:47.635219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdf711-06b8-42a0-831d-940d91db8791 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:47.652791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1f98e924-36a2-44d8-9f0a-f7404c56c2b1 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:47.654205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1f98e924-36a2-44d8-9f0a-f7404c56c2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:47.700272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b90d6e-0219-4c05-a450-6b8682675698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:47.701919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b90d6e-0219-4c05-a450-6b8682675698 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:47.702202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b90d6e-0219-4c05-a450-6b8682675698 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:47.702476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b90d6e-0219-4c05-a450-6b8682675698 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:47.703423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b90d6e-0219-4c05-a450-6b8682675698 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:47.703727 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b90d6e-0219-4c05-a450-6b8682675698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:47.705334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b90d6e-0219-4c05-a450-6b8682675698 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:47.727106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 170/342] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:47.727690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b90d6e-0219-4c05-a450-6b8682675698 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:47.753334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 173/343] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:47.754261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98e924-36a2-44d8-9f0a-f7404c56c2b1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:47.759365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:47.811403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:47.875738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b90d6e-0219-4c05-a450-6b8682675698 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:47.932845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823c31c-ed91-4a55-9016-96459dbb446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:47.933448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823c31c-ed91-4a55-9016-96459dbb446c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:47.933726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823c31c-ed91-4a55-9016-96459dbb446c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:47.934005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823c31c-ed91-4a55-9016-96459dbb446c None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:47.934674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823c31c-ed91-4a55-9016-96459dbb446c None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:47.934978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823c31c-ed91-4a55-9016-96459dbb446c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:47.936352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823c31c-ed91-4a55-9016-96459dbb446c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:47.943726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98e924-36a2-44d8-9f0a-f7404c56c2b1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:47.945716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 171/344] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:47.952226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823c31c-ed91-4a55-9016-96459dbb446c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:47.997525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059f997-44e0-4e46-8c6e-be2c0dc20531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:47.998064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059f997-44e0-4e46-8c6e-be2c0dc20531 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:47.998340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059f997-44e0-4e46-8c6e-be2c0dc20531 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:47.998611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059f997-44e0-4e46-8c6e-be2c0dc20531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.058467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:48.083709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823c31c-ed91-4a55-9016-96459dbb446c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:48.121836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059f997-44e0-4e46-8c6e-be2c0dc20531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:48.123051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d6287-4cb3-4e5d-9fcb-f8d5aec2c2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:48.123511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059f997-44e0-4e46-8c6e-be2c0dc20531 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:48.123706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d6287-4cb3-4e5d-9fcb-f8d5aec2c2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.123938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d6287-4cb3-4e5d-9fcb-f8d5aec2c2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.124210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d6287-4cb3-4e5d-9fcb-f8d5aec2c2f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.184531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:48.204009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d6287-4cb3-4e5d-9fcb-f8d5aec2c2f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:48.206196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d6287-4cb3-4e5d-9fcb-f8d5aec2c2f6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:48.297267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:48.300469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 174/345] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:48.301377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059f997-44e0-4e46-8c6e-be2c0dc20531 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:48.311057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 172/346] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 10:42:48.312097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d6287-4cb3-4e5d-9fcb-f8d5aec2c2f6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:48.420584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d6287-4cb3-4e5d-9fcb-f8d5aec2c2f6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:48.457781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2106e7-7b7c-4095-bd20-175e10063159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:48.458427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2106e7-7b7c-4095-bd20-175e10063159 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.458701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2106e7-7b7c-4095-bd20-175e10063159 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.458973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2106e7-7b7c-4095-bd20-175e10063159 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.460118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2106e7-7b7c-4095-bd20-175e10063159 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:48.461628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2106e7-7b7c-4095-bd20-175e10063159 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:48.463261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 173/347] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:48 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:48.464232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2106e7-7b7c-4095-bd20-175e10063159 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:48.525223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:48.548747 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059f997-44e0-4e46-8c6e-be2c0dc20531 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:48.577627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2106e7-7b7c-4095-bd20-175e10063159 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:48.613920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85f581-244b-438b-a0b0-f0eb9f799acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:48.614456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd59f24-7769-4eab-a6ce-ce39d9811db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:48.614894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85f581-244b-438b-a0b0-f0eb9f799acf None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.614894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85f581-244b-438b-a0b0-f0eb9f799acf None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.615108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85f581-244b-438b-a0b0-f0eb9f799acf None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.615181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd59f24-7769-4eab-a6ce-ce39d9811db2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.615373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd59f24-7769-4eab-a6ce-ce39d9811db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.615742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd59f24-7769-4eab-a6ce-ce39d9811db2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.616444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85f581-244b-438b-a0b0-f0eb9f799acf None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:48.618037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85f581-244b-438b-a0b0-f0eb9f799acf None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:48.619690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 174/348] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:48 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:48.620463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85f581-244b-438b-a0b0-f0eb9f799acf None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:48.624665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:48.628351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd59f24-7769-4eab-a6ce-ce39d9811db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-7cd59f24-7769-4eab-a6ce-ce39d9811db2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:48.634235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:48.635165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:48.635987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 175/349] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:48.637184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d690b73-8610-439f-8989-086e90b650e9 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.638954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d690b73-8610-439f-8989-086e90b650e9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.638954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d690b73-8610-439f-8989-086e90b650e9 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.639303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 176/350] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:48.639963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d690b73-8610-439f-8989-086e90b650e9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:48.739386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85f581-244b-438b-a0b0-f0eb9f799acf None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:48.757293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:48.776193 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d690b73-8610-439f-8989-086e90b650e9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:48.784554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bd5e4-f18e-4f08-8448-d5af0b4dd36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:48.785822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bd5e4-f18e-4f08-8448-d5af0b4dd36e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.786101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bd5e4-f18e-4f08-8448-d5af0b4dd36e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.786376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bd5e4-f18e-4f08-8448-d5af0b4dd36e None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.793911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bd5e4-f18e-4f08-8448-d5af0b4dd36e None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:48.795604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bd5e4-f18e-4f08-8448-d5af0b4dd36e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:48.797480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 175/351] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:48 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:48.801787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bd5e4-f18e-4f08-8448-d5af0b4dd36e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:48.822509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362024f3-7fc3-49d0-8c1f-57e6966f57fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:48.823559 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362024f3-7fc3-49d0-8c1f-57e6966f57fb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.823997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362024f3-7fc3-49d0-8c1f-57e6966f57fb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.824408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362024f3-7fc3-49d0-8c1f-57e6966f57fb None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.825757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362024f3-7fc3-49d0-8c1f-57e6966f57fb None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:48.827607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362024f3-7fc3-49d0-8c1f-57e6966f57fb None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:48.830372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 177/352] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:48 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:48.831592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e024a2c-9ada-4615-a88b-913d98bd9230 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.831882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e024a2c-9ada-4615-a88b-913d98bd9230 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.832325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e024a2c-9ada-4615-a88b-913d98bd9230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.854241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3e024a2c-9ada-4615-a88b-913d98bd9230 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:48.855785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3e024a2c-9ada-4615-a88b-913d98bd9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:48.928295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:48.932720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bd5e4-f18e-4f08-8448-d5af0b4dd36e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:48.938696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 178/353] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:48.940274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29490901-e233-4b17-a641-5e8d29c3597a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.940796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29490901-e233-4b17-a641-5e8d29c3597a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.941297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29490901-e233-4b17-a641-5e8d29c3597a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.942901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 179/354] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:48.943848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29490901-e233-4b17-a641-5e8d29c3597a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:48.978026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73fb43-8b35-4990-a754-321bd0dc3e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:48.979108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73fb43-8b35-4990-a754-321bd0dc3e42 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:48.979595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73fb43-8b35-4990-a754-321bd0dc3e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:48.980083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73fb43-8b35-4990-a754-321bd0dc3e42 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:48.980945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73fb43-8b35-4990-a754-321bd0dc3e42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:48.982032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73fb43-8b35-4990-a754-321bd0dc3e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:48.982848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73fb43-8b35-4990-a754-321bd0dc3e42 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:48.994691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 176/355] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:48.996087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73fb43-8b35-4990-a754-321bd0dc3e42 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:49.054732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:49.066921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29490901-e233-4b17-a641-5e8d29c3597a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:49.105958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47b4e1-cf07-487b-b1e4-33e61bd45051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:49.106571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47b4e1-cf07-487b-b1e4-33e61bd45051 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.106848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47b4e1-cf07-487b-b1e4-33e61bd45051 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.107114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47b4e1-cf07-487b-b1e4-33e61bd45051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.107838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47b4e1-cf07-487b-b1e4-33e61bd45051 None admin] RBAC: Adding query filter params (group.id=8eede211d23743ed98fb3e4b7bf22efe, role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:49.108166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47b4e1-cf07-487b-b1e4-33e61bd45051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:49.109741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47b4e1-cf07-487b-b1e4-33e61bd45051 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:49.120834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 180/356] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:42:48 2022] GET /identity/v3/role_assignments?group.id=8eede211d23743ed98fb3e4b7bf22efe&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=3518baf98ba54a28a9c79ec287469aff => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:49.120834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f0f1f-c6f4-4dbc-a4f6-84bdbae1a00b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.120834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f0f1f-c6f4-4dbc-a4f6-84bdbae1a00b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.120834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f0f1f-c6f4-4dbc-a4f6-84bdbae1a00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.142761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a67f0f1f-c6f4-4dbc-a4f6-84bdbae1a00b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:49.144195 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a67f0f1f-c6f4-4dbc-a4f6-84bdbae1a00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:49.148781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73fb43-8b35-4990-a754-321bd0dc3e42 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:49.202030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e03ec-d7b8-4d6c-bc25-6e1359ddfdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:49.203301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e03ec-d7b8-4d6c-bc25-6e1359ddfdda None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.203788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e03ec-d7b8-4d6c-bc25-6e1359ddfdda None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.204920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e03ec-d7b8-4d6c-bc25-6e1359ddfdda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.206555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e03ec-d7b8-4d6c-bc25-6e1359ddfdda None admin] RBAC: Adding query filter params (group.id=8eede211d23743ed98fb3e4b7bf22efe, role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:49.207223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e03ec-d7b8-4d6c-bc25-6e1359ddfdda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:49.211769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e03ec-d7b8-4d6c-bc25-6e1359ddfdda None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:49.219688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 177/357] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:42:48 2022] GET /identity/v3/role_assignments?group.id=8eede211d23743ed98fb3e4b7bf22efe&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:49.221564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e03ec-d7b8-4d6c-bc25-6e1359ddfdda None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:49.278337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:49.306845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 181/358] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:49.307767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f0f1f-c6f4-4dbc-a4f6-84bdbae1a00b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:49.414189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e03ec-d7b8-4d6c-bc25-6e1359ddfdda None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:49.417360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:49.459662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503c1b2-d23c-4066-9d76-f5dfe3e95fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:49.460398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503c1b2-d23c-4066-9d76-f5dfe3e95fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.460750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503c1b2-d23c-4066-9d76-f5dfe3e95fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.461059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503c1b2-d23c-4066-9d76-f5dfe3e95fd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.483966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f0f1f-c6f4-4dbc-a4f6-84bdbae1a00b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:49.528054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babd796-b163-4185-911d-8dfa2c6b8e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:49.528703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd796-b163-4185-911d-8dfa2c6b8e06 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.529053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd796-b163-4185-911d-8dfa2c6b8e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.529325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd796-b163-4185-911d-8dfa2c6b8e06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.544546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503c1b2-d23c-4066-9d76-f5dfe3e95fd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:49.546061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503c1b2-d23c-4066-9d76-f5dfe3e95fd6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:49.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babd796-b163-4185-911d-8dfa2c6b8e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3babd796-b163-4185-911d-8dfa2c6b8e06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:49.567628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:49.568506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 10:42:49.569353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:49.576325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 182/359] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:49.578365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6083f55-dc17-4134-925b-01ae5e6bf5ac None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.578869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6083f55-dc17-4134-925b-01ae5e6bf5ac None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.579426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6083f55-dc17-4134-925b-01ae5e6bf5ac None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.581860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 183/360] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:49.582811 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6083f55-dc17-4134-925b-01ae5e6bf5ac None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:49.647646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:49.666887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 178/361] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:49.668517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919016a-4d6f-4a24-a3d2-0f9b2aac282d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.669050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919016a-4d6f-4a24-a3d2-0f9b2aac282d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.669519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919016a-4d6f-4a24-a3d2-0f9b2aac282d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.683339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:49.703537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2919016a-4d6f-4a24-a3d2-0f9b2aac282d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:49.705146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2919016a-4d6f-4a24-a3d2-0f9b2aac282d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:49.709143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6083f55-dc17-4134-925b-01ae5e6bf5ac None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:49.750768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e99871-c24e-416f-9d55-51447d61aa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:49.751507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e99871-c24e-416f-9d55-51447d61aa57 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.751924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e99871-c24e-416f-9d55-51447d61aa57 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.752244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e99871-c24e-416f-9d55-51447d61aa57 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.752952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e99871-c24e-416f-9d55-51447d61aa57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:49.753256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e99871-c24e-416f-9d55-51447d61aa57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:49.754651 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e99871-c24e-416f-9d55-51447d61aa57 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:49.764629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 184/362] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:49.765581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e99871-c24e-416f-9d55-51447d61aa57 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:49.796600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 179/363] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:49.798171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919016a-4d6f-4a24-a3d2-0f9b2aac282d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:49.916590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e99871-c24e-416f-9d55-51447d61aa57 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:49.929944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:49.936973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919016a-4d6f-4a24-a3d2-0f9b2aac282d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:49.959318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d8b0f-5bdd-47ec-940b-5d6499a38011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:49.960062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d8b0f-5bdd-47ec-940b-5d6499a38011 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.960416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d8b0f-5bdd-47ec-940b-5d6499a38011 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.960760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d8b0f-5bdd-47ec-940b-5d6499a38011 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.975081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d8b0f-5bdd-47ec-940b-5d6499a38011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:49.980529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe29a03-71ef-4a63-ab36-f4efc6fa609f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-787d8b0f-5bdd-47ec-940b-5d6499a38011 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:49.981141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:49.982216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:49.983241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:49.984043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe29a03-71ef-4a63-ab36-f4efc6fa609f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:49.984043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe29a03-71ef-4a63-ab36-f4efc6fa609f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:49.984043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe29a03-71ef-4a63-ab36-f4efc6fa609f None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:49.984880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe29a03-71ef-4a63-ab36-f4efc6fa609f None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:49.986294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 185/364] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:49.987487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d8b0f-5bdd-47ec-940b-5d6499a38011 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:49.987614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe29a03-71ef-4a63-ab36-f4efc6fa609f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:49.995855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 180/365] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:49 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:49.996711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe29a03-71ef-4a63-ab36-f4efc6fa609f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:50.007007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:50.107655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d8b0f-5bdd-47ec-940b-5d6499a38011 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:50.147235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dd90e-1603-44b1-93fe-4e50afffc612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:50.147874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dd90e-1603-44b1-93fe-4e50afffc612 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:50.148155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dd90e-1603-44b1-93fe-4e50afffc612 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:50.148429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dd90e-1603-44b1-93fe-4e50afffc612 None admin] PATH_INFO: `/v3/users/a48b886f49bb4111919a535126b3a7b2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:50.149969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7dd90e-1603-44b1-93fe-4e50afffc612 None admin] RBAC: Authorizing `identity:get_user(user_id=a48b886f49bb4111919a535126b3a7b2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:50.151591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7dd90e-1603-44b1-93fe-4e50afffc612 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:50.153386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 186/366] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:42:49 2022] GET /identity/v3/users/a48b886f49bb4111919a535126b3a7b2 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:50.153986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dd90e-1603-44b1-93fe-4e50afffc612 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:50.205954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:50.274031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe29a03-71ef-4a63-ab36-f4efc6fa609f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:50.287232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dd90e-1603-44b1-93fe-4e50afffc612 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:50.317285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b2075-a534-409f-baab-1f6d1b6fd7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:50.317902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b2075-a534-409f-baab-1f6d1b6fd7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:50.318179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b2075-a534-409f-baab-1f6d1b6fd7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:50.318444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b2075-a534-409f-baab-1f6d1b6fd7b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:50.319103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b2075-a534-409f-baab-1f6d1b6fd7b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:50.319405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b2075-a534-409f-baab-1f6d1b6fd7b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:50.320776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b2075-a534-409f-baab-1f6d1b6fd7b0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:50.338830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 181/367] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:50.339676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b2075-a534-409f-baab-1f6d1b6fd7b0 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:50.339883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc3bd7-7c98-4d4c-8ef3-60df9fcee9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:50.341402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc3bd7-7c98-4d4c-8ef3-60df9fcee9ea None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:50.341894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc3bd7-7c98-4d4c-8ef3-60df9fcee9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:50.342706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc3bd7-7c98-4d4c-8ef3-60df9fcee9ea None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:50.345649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc3bd7-7c98-4d4c-8ef3-60df9fcee9ea None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:50.347354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc3bd7-7c98-4d4c-8ef3-60df9fcee9ea None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:50.350092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 187/368] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:50 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:50.351086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc3bd7-7c98-4d4c-8ef3-60df9fcee9ea None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:50.422991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:50.477115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b2075-a534-409f-baab-1f6d1b6fd7b0 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:50.499412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc3bd7-7c98-4d4c-8ef3-60df9fcee9ea None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:50.509146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:50.514936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327207-a2f7-42fa-9235-ee3db0d63d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:50.515574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327207-a2f7-42fa-9235-ee3db0d63d0d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:50.515856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327207-a2f7-42fa-9235-ee3db0d63d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:50.516126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327207-a2f7-42fa-9235-ee3db0d63d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:50.517515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69327207-a2f7-42fa-9235-ee3db0d63d0d None admin] RBAC: Adding query filter params (role.id=fdf0d588fb134183911419bbaa627d22, user.id=a48b886f49bb4111919a535126b3a7b2, scope.system=all) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:50.517831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69327207-a2f7-42fa-9235-ee3db0d63d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:50.519706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69327207-a2f7-42fa-9235-ee3db0d63d0d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:50.534365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 182/369] 10.222.0.50 () {56 vars in 1308 bytes} [Thu Sep 15 10:42:50 2022] GET /identity/v3/role_assignments?user.id=a48b886f49bb4111919a535126b3a7b2&scope.system=all&role.id=fdf0d588fb134183911419bbaa627d22 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:50.535053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327207-a2f7-42fa-9235-ee3db0d63d0d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:50.550631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63039-0c47-4027-abe3-2e2aaaaa99c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:50.551687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63039-0c47-4027-abe3-2e2aaaaa99c7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:50.552166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63039-0c47-4027-abe3-2e2aaaaa99c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:50.552657 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63039-0c47-4027-abe3-2e2aaaaa99c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:50.566562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e63039-0c47-4027-abe3-2e2aaaaa99c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-12e63039-0c47-4027-abe3-2e2aaaaa99c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:50.572129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:50.573115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:50.573920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:50.576710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 188/370] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:42:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:50.578180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63039-0c47-4027-abe3-2e2aaaaa99c7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:50.698079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327207-a2f7-42fa-9235-ee3db0d63d0d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:50.699741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63039-0c47-4027-abe3-2e2aaaaa99c7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:50.731122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:50.744683 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea00c0-1c3c-4aa3-82d1-cab1606a29df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:50.745862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea00c0-1c3c-4aa3-82d1-cab1606a29df None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:50.746346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea00c0-1c3c-4aa3-82d1-cab1606a29df None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:50.746822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea00c0-1c3c-4aa3-82d1-cab1606a29df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:50.747781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea00c0-1c3c-4aa3-82d1-cab1606a29df None admin] RBAC: Adding query filter params (group.id=8eede211d23743ed98fb3e4b7bf22efe, role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=caedb3d1f7f94504824174418c72d3df) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:50.748327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea00c0-1c3c-4aa3-82d1-cab1606a29df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:50.750375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea00c0-1c3c-4aa3-82d1-cab1606a29df None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:50.752115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b43c4b-31e2-4d03-929f-bca3272ffa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:50.752740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43c4b-31e2-4d03-929f-bca3272ffa6c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:50.753047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43c4b-31e2-4d03-929f-bca3272ffa6c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:50.753322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43c4b-31e2-4d03-929f-bca3272ffa6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:50.758698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 189/371] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:42:50 2022] GET /identity/v3/role_assignments?group.id=8eede211d23743ed98fb3e4b7bf22efe&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:50.759847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea00c0-1c3c-4aa3-82d1-cab1606a29df None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:50.793949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:50.880177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b43c4b-31e2-4d03-929f-bca3272ffa6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:50.881700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b43c4b-31e2-4d03-929f-bca3272ffa6c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:50.949883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea00c0-1c3c-4aa3-82d1-cab1606a29df None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:51.035669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739efa5-2d33-40fd-afef-bfcc1c02d4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:51.036226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739efa5-2d33-40fd-afef-bfcc1c02d4ec None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:51.036511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739efa5-2d33-40fd-afef-bfcc1c02d4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:51.036785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739efa5-2d33-40fd-afef-bfcc1c02d4ec None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:51.037484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739efa5-2d33-40fd-afef-bfcc1c02d4ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:51.037789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739efa5-2d33-40fd-afef-bfcc1c02d4ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:51.039227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739efa5-2d33-40fd-afef-bfcc1c02d4ec None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:51.058779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 190/372] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:51.065517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 183/373] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:51.066552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739efa5-2d33-40fd-afef-bfcc1c02d4ec None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:51.071242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b43c4b-31e2-4d03-929f-bca3272ffa6c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:51.126927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:51.254938 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:51.272879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739efa5-2d33-40fd-afef-bfcc1c02d4ec None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:51.276035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b43c4b-31e2-4d03-929f-bca3272ffa6c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:51.329167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f22d97-a0ae-488e-9b30-471d2310baa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:51.329978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f22d97-a0ae-488e-9b30-471d2310baa6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:51.330350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f22d97-a0ae-488e-9b30-471d2310baa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:51.333560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141286e-c050-4172-96d5-b44dd8e192d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:51.334217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141286e-c050-4172-96d5-b44dd8e192d1 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:51.334500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141286e-c050-4172-96d5-b44dd8e192d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:51.334677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f22d97-a0ae-488e-9b30-471d2310baa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:51.334774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141286e-c050-4172-96d5-b44dd8e192d1 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:51.335921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141286e-c050-4172-96d5-b44dd8e192d1 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:51.337493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141286e-c050-4172-96d5-b44dd8e192d1 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:51.339130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 184/374] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:51 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:51.345202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141286e-c050-4172-96d5-b44dd8e192d1 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:51.468959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f22d97-a0ae-488e-9b30-471d2310baa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:51.470484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f22d97-a0ae-488e-9b30-471d2310baa6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:51.529343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:51.550393 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141286e-c050-4172-96d5-b44dd8e192d1 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:51.618236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757eb9c8-4ee2-4b46-9534-2a05c98afaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:51.618817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757eb9c8-4ee2-4b46-9534-2a05c98afaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:51.619166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757eb9c8-4ee2-4b46-9534-2a05c98afaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:51.619450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757eb9c8-4ee2-4b46-9534-2a05c98afaa5 None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:51.625497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757eb9c8-4ee2-4b46-9534-2a05c98afaa5 None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:51.627541 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757eb9c8-4ee2-4b46-9534-2a05c98afaa5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:51.629590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 185/375] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:51 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:51.631191 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:51.634964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757eb9c8-4ee2-4b46-9534-2a05c98afaa5 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:51.643504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 191/376] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:51.649151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f22d97-a0ae-488e-9b30-471d2310baa6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:51.761009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f22d97-a0ae-488e-9b30-471d2310baa6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:51.766800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757eb9c8-4ee2-4b46-9534-2a05c98afaa5 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:51.804137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47d8c4-ed7e-432b-ae41-10f08b3a5af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:51.804892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47d8c4-ed7e-432b-ae41-10f08b3a5af3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:51.805002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5e994-db80-491f-b1ed-435a1999073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:51.805181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47d8c4-ed7e-432b-ae41-10f08b3a5af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:51.805449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47d8c4-ed7e-432b-ae41-10f08b3a5af3 None admin] PATH_INFO: `/v3/projects/70f07107ef09454996be197ead3ddf41` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:51.805548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5e994-db80-491f-b1ed-435a1999073e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:51.805783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5e994-db80-491f-b1ed-435a1999073e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:51.806058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5e994-db80-491f-b1ed-435a1999073e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:51.806592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47d8c4-ed7e-432b-ae41-10f08b3a5af3 None admin] RBAC: Authorizing `identity:get_project(project_id=70f07107ef09454996be197ead3ddf41)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:51.806782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5e994-db80-491f-b1ed-435a1999073e None admin] RBAC: Adding query filter params (group.id=8eede211d23743ed98fb3e4b7bf22efe, role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=caedb3d1f7f94504824174418c72d3df) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:51.807098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5e994-db80-491f-b1ed-435a1999073e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:51.808053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47d8c4-ed7e-432b-ae41-10f08b3a5af3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:51.808520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5e994-db80-491f-b1ed-435a1999073e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:51.810006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 192/377] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:51 2022] GET /identity/v3/projects/70f07107ef09454996be197ead3ddf41 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:51.814750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 186/378] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:42:51 2022] GET /identity/v3/role_assignments?group.id=8eede211d23743ed98fb3e4b7bf22efe&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=3518baf98ba54a28a9c79ec287469aff => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:51.815664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47d8c4-ed7e-432b-ae41-10f08b3a5af3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:51.901111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:51.966832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47d8c4-ed7e-432b-ae41-10f08b3a5af3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:52.018087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b50f76-015b-45d2-a507-2511df00a542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:52.019167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b50f76-015b-45d2-a507-2511df00a542 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:52.019662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b50f76-015b-45d2-a507-2511df00a542 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:52.020152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b50f76-015b-45d2-a507-2511df00a542 None admin] PATH_INFO: `/v3/groups/c20709d488a44293b4d73fa0787c394b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:52.031586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b50f76-015b-45d2-a507-2511df00a542 None admin] RBAC: Authorizing `identity:get_group(group_id=c20709d488a44293b4d73fa0787c394b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:52.033466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b50f76-015b-45d2-a507-2511df00a542 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:52.036120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 193/379] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:51 2022] GET /identity/v3/groups/c20709d488a44293b4d73fa0787c394b => generated 229 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:52.042047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5e994-db80-491f-b1ed-435a1999073e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:52.058136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6e215-b327-4af9-9df3-481b0e161a14 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:52.059171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6e215-b327-4af9-9df3-481b0e161a14 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:52.059660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6e215-b327-4af9-9df3-481b0e161a14 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:52.061726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 194/380] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:52.075181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5ad6-05f7-47e6-93d7-6db61e5de334 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:52.075836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5ad6-05f7-47e6-93d7-6db61e5de334 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:52.076317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5ad6-05f7-47e6-93d7-6db61e5de334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:52.082799 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:52.110299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5c3d5ad6-05f7-47e6-93d7-6db61e5de334 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:52.112521 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5c3d5ad6-05f7-47e6-93d7-6db61e5de334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:52.201008 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 195/381] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:52.221474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb245c-d3cd-4e40-addd-47a6b07d8f64 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:52.222395 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb245c-d3cd-4e40-addd-47a6b07d8f64 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:52.223162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb245c-d3cd-4e40-addd-47a6b07d8f64 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:52.225527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 196/382] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:52.230590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af5d4d-aa65-4b1a-8833-db91a0628b66 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:52.231484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af5d4d-aa65-4b1a-8833-db91a0628b66 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:52.231985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af5d4d-aa65-4b1a-8833-db91a0628b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:52.235063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5e994-db80-491f-b1ed-435a1999073e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:52.268459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f2af5d4d-aa65-4b1a-8833-db91a0628b66 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:52.269706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f2af5d4d-aa65-4b1a-8833-db91a0628b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:52.299842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb918b4-cddd-4552-b639-4b7c9c801394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:52.300470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb918b4-cddd-4552-b639-4b7c9c801394 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:52.300754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb918b4-cddd-4552-b639-4b7c9c801394 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:52.301075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb918b4-cddd-4552-b639-4b7c9c801394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:52.301819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb918b4-cddd-4552-b639-4b7c9c801394 None admin] RBAC: Adding query filter params (group.id=c20709d488a44293b4d73fa0787c394b, role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=70f07107ef09454996be197ead3ddf41) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:52.302132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb918b4-cddd-4552-b639-4b7c9c801394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:52.303650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb918b4-cddd-4552-b639-4b7c9c801394 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:52.317219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 187/383] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:42:52 2022] GET /identity/v3/role_assignments?group.id=c20709d488a44293b4d73fa0787c394b&scope.project.id=70f07107ef09454996be197ead3ddf41&role.id=0be9777002c641b4a47f78df647cc173 => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:52.359536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:52.385614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 197/384] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:52.528199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb918b4-cddd-4552-b639-4b7c9c801394 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:52.628101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:52.645764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb918b4-cddd-4552-b639-4b7c9c801394 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:52.689187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a80a3-2ce0-4d59-a197-fa249bcbe5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:52.690208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a80a3-2ce0-4d59-a197-fa249bcbe5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:52.690695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a80a3-2ce0-4d59-a197-fa249bcbe5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:52.691167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a80a3-2ce0-4d59-a197-fa249bcbe5b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:52.702933 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a80a3-2ce0-4d59-a197-fa249bcbe5b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-3f3a80a3-2ce0-4d59-a197-fa249bcbe5b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:52.708937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:52.709852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:52.710665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:52.713309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 188/385] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:52.721965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af5d4d-aa65-4b1a-8833-db91a0628b66 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:52.840986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af5d4d-aa65-4b1a-8833-db91a0628b66 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:52.862143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a80a3-2ce0-4d59-a197-fa249bcbe5b2 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:52.892839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce5284-5646-493d-85db-d933dc52bb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:52.892839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce5284-5646-493d-85db-d933dc52bb49 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:52.892839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce5284-5646-493d-85db-d933dc52bb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:52.892839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce5284-5646-493d-85db-d933dc52bb49 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:52.893251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce5284-5646-493d-85db-d933dc52bb49 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:52.893251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce5284-5646-493d-85db-d933dc52bb49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:52.893761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce5284-5646-493d-85db-d933dc52bb49 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:52.912076 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 198/386] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:52.918372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce5284-5646-493d-85db-d933dc52bb49 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:52.991836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:53.006026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a80a3-2ce0-4d59-a197-fa249bcbe5b2 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:53.075865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae1a65-81ba-4302-b226-92c91c2ffc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:53.077369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae1a65-81ba-4302-b226-92c91c2ffc9a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.078118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae1a65-81ba-4302-b226-92c91c2ffc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.078591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae1a65-81ba-4302-b226-92c91c2ffc9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.099251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae1a65-81ba-4302-b226-92c91c2ffc9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-7aae1a65-81ba-4302-b226-92c91c2ffc9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:53.105262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:53.106183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:53.106983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:53.110897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce5284-5646-493d-85db-d933dc52bb49 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:53.113965 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 189/387] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:53.120847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae1a65-81ba-4302-b226-92c91c2ffc9a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:53.136428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:53.185013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f45be7-b4f6-4b18-9b09-b0bea7852cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:53.186039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f45be7-b4f6-4b18-9b09-b0bea7852cec None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.187389 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f45be7-b4f6-4b18-9b09-b0bea7852cec None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.189642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f45be7-b4f6-4b18-9b09-b0bea7852cec None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.191235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f45be7-b4f6-4b18-9b09-b0bea7852cec None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:53.194957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f45be7-b4f6-4b18-9b09-b0bea7852cec None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:53.203507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 199/388] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:52 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:53.209468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f45be7-b4f6-4b18-9b09-b0bea7852cec None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:53.325126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae1a65-81ba-4302-b226-92c91c2ffc9a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:53.368373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7b450-76d9-4801-936c-b9389b6c1400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:53.369416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7b450-76d9-4801-936c-b9389b6c1400 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.369910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7b450-76d9-4801-936c-b9389b6c1400 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.370385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7b450-76d9-4801-936c-b9389b6c1400 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.371419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7b450-76d9-4801-936c-b9389b6c1400 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:53.371927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7b450-76d9-4801-936c-b9389b6c1400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:53.374468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7b450-76d9-4801-936c-b9389b6c1400 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:53.389309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 190/389] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:53.391301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7b450-76d9-4801-936c-b9389b6c1400 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:53.466966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f45be7-b4f6-4b18-9b09-b0bea7852cec None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:53.472515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:53.506234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:53.513650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7b450-76d9-4801-936c-b9389b6c1400 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:53.554757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a3745-4a4a-4a85-9b35-e55805410175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:53.555755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a3745-4a4a-4a85-9b35-e55805410175 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.556269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a3745-4a4a-4a85-9b35-e55805410175 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.557441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a3745-4a4a-4a85-9b35-e55805410175 None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.558883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a3745-4a4a-4a85-9b35-e55805410175 None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:53.561733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a3745-4a4a-4a85-9b35-e55805410175 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:53.568708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 191/390] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:53 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:53.570245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a3745-4a4a-4a85-9b35-e55805410175 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:53.585553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc457d7e-85b0-4fde-af15-67dbfc21801f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:53.586222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc457d7e-85b0-4fde-af15-67dbfc21801f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.586505 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc457d7e-85b0-4fde-af15-67dbfc21801f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.586780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc457d7e-85b0-4fde-af15-67dbfc21801f None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.593510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc457d7e-85b0-4fde-af15-67dbfc21801f None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:53.595697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc457d7e-85b0-4fde-af15-67dbfc21801f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:53.597587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 200/391] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:53 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:53.603161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc457d7e-85b0-4fde-af15-67dbfc21801f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:53.687883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a3745-4a4a-4a85-9b35-e55805410175 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:53.728147 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60434e9f-1f9b-41c3-9b38-45671a1c6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:53.729167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60434e9f-1f9b-41c3-9b38-45671a1c6164 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.729655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60434e9f-1f9b-41c3-9b38-45671a1c6164 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.730145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60434e9f-1f9b-41c3-9b38-45671a1c6164 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.732630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60434e9f-1f9b-41c3-9b38-45671a1c6164 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:53.734432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60434e9f-1f9b-41c3-9b38-45671a1c6164 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:53.740605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 192/392] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:53 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:53.742536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60434e9f-1f9b-41c3-9b38-45671a1c6164 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:53.747141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:53.854032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc457d7e-85b0-4fde-af15-67dbfc21801f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:53.854564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60434e9f-1f9b-41c3-9b38-45671a1c6164 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:53.892740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69d599-0276-4e8d-b883-992eeb86b7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:53.893817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d599-0276-4e8d-b883-992eeb86b7bc None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.894285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d599-0276-4e8d-b883-992eeb86b7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.894842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d599-0276-4e8d-b883-992eeb86b7bc None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/groups/8eede211d23743ed98fb3e4b7bf22efe/roles/3518baf98ba54a28a9c79ec287469aff` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.895212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398dbf2c-794a-4d84-9aed-72c8dad062bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:53.895953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dbf2c-794a-4d84-9aed-72c8dad062bd None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.896306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dbf2c-794a-4d84-9aed-72c8dad062bd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.896650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dbf2c-794a-4d84-9aed-72c8dad062bd None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/groups/8eede211d23743ed98fb3e4b7bf22efe/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.898294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69d599-0276-4e8d-b883-992eeb86b7bc None admin] RBAC: Authorizing `identity:create_grant(project_id=a9cd5f1606ad462482bd725a3a3537a3, group_id=8eede211d23743ed98fb3e4b7bf22efe, role_id=3518baf98ba54a28a9c79ec287469aff)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:53.898752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398dbf2c-794a-4d84-9aed-72c8dad062bd None admin] RBAC: Authorizing `identity:create_grant(project_id=a9cd5f1606ad462482bd725a3a3537a3, group_id=8eede211d23743ed98fb3e4b7bf22efe, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:53.900139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69d599-0276-4e8d-b883-992eeb86b7bc None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:53.900297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398dbf2c-794a-4d84-9aed-72c8dad062bd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:53.916901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 193/393] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Sep 15 10:42:53 2022] PUT /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/groups/8eede211d23743ed98fb3e4b7bf22efe/roles/3518baf98ba54a28a9c79ec287469aff => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:42:53.918104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 201/394] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Sep 15 10:42:53 2022] PUT /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3/groups/8eede211d23743ed98fb3e4b7bf22efe/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:53.918826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7613dd-b2df-470d-85dd-c5a4f0d6ca61 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.918934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7613dd-b2df-470d-85dd-c5a4f0d6ca61 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.925444 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7613dd-b2df-470d-85dd-c5a4f0d6ca61 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.926978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 202/395] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:53.938858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529438e-674f-47da-8b6c-432259077ab2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:53.939428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529438e-674f-47da-8b6c-432259077ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:53.939885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529438e-674f-47da-8b6c-432259077ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:53.970757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-6529438e-674f-47da-8b6c-432259077ab2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:53.972318 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6529438e-674f-47da-8b6c-432259077ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:54.056780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:54.068584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 194/396] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:54.407614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7613dd-b2df-470d-85dd-c5a4f0d6ca61 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:54.471036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:54.548592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7613dd-b2df-470d-85dd-c5a4f0d6ca61 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:54.592473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9c197-7a82-4502-9f3f-6814ac553df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:54.593170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c197-7a82-4502-9f3f-6814ac553df4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:54.593451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c197-7a82-4502-9f3f-6814ac553df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:54.593722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c197-7a82-4502-9f3f-6814ac553df4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:54.608122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9c197-7a82-4502-9f3f-6814ac553df4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-1eb9c197-7a82-4502-9f3f-6814ac553df4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:54.613945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:54.614861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:54.616188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 203/397] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:54.621186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529438e-674f-47da-8b6c-432259077ab2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:54.753134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529438e-674f-47da-8b6c-432259077ab2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:54.805061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58540d9e-17e5-429b-9314-9b75afdf8287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:54.805673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58540d9e-17e5-429b-9314-9b75afdf8287 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:54.805955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58540d9e-17e5-429b-9314-9b75afdf8287 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:54.806236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58540d9e-17e5-429b-9314-9b75afdf8287 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:54.806906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58540d9e-17e5-429b-9314-9b75afdf8287 None admin] RBAC: Adding query filter params (name=member) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:54.807214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58540d9e-17e5-429b-9314-9b75afdf8287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:54.808615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58540d9e-17e5-429b-9314-9b75afdf8287 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:54.820431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 195/398] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:54.826051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9c197-7a82-4502-9f3f-6814ac553df4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:54.875252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8408b7-0ebc-4a84-905f-29217132bc6d None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:54.875558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8408b7-0ebc-4a84-905f-29217132bc6d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:54.875865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8408b7-0ebc-4a84-905f-29217132bc6d None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:54.877543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 196/399] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:54.886226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61b4f1-6e7c-4cf5-93a8-2ba98c5bfeba None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:54.886523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61b4f1-6e7c-4cf5-93a8-2ba98c5bfeba None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:54.886803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61b4f1-6e7c-4cf5-93a8-2ba98c5bfeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:54.918132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-6c61b4f1-6e7c-4cf5-93a8-2ba98c5bfeba None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:54.919345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6c61b4f1-6e7c-4cf5-93a8-2ba98c5bfeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:55.014564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9c197-7a82-4502-9f3f-6814ac553df4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:55.020123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:55.029099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:55.061322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 197/400] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:55.075000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5d046-c419-4ba9-b5f0-cadf1c132147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:55.075649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5d046-c419-4ba9-b5f0-cadf1c132147 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:55.075936 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5d046-c419-4ba9-b5f0-cadf1c132147 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:55.076218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5d046-c419-4ba9-b5f0-cadf1c132147 None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:55.077373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5d046-c419-4ba9-b5f0-cadf1c132147 None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:55.079417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5d046-c419-4ba9-b5f0-cadf1c132147 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:55.081465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 204/401] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:54 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:55.090094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61b4f1-6e7c-4cf5-93a8-2ba98c5bfeba None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:55.153777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07670fb-caf2-4aab-a5d0-e37763da73e3 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:55.154120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07670fb-caf2-4aab-a5d0-e37763da73e3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:55.154342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07670fb-caf2-4aab-a5d0-e37763da73e3 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:55.155841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 205/402] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:55.161183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a3c80-0620-4977-9d0e-29b7e68a1f54 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:55.161465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a3c80-0620-4977-9d0e-29b7e68a1f54 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:55.161737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a3c80-0620-4977-9d0e-29b7e68a1f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:55.185667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3d6a3c80-0620-4977-9d0e-29b7e68a1f54 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:55.186984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3d6a3c80-0620-4977-9d0e-29b7e68a1f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:55.250654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61b4f1-6e7c-4cf5-93a8-2ba98c5bfeba None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:55.306583 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768686a-32ff-4b77-9158-9ad2a0d23095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:55.307137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768686a-32ff-4b77-9158-9ad2a0d23095 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:55.307417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768686a-32ff-4b77-9158-9ad2a0d23095 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:55.307690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768686a-32ff-4b77-9158-9ad2a0d23095 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:55.308900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768686a-32ff-4b77-9158-9ad2a0d23095 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:55.310422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768686a-32ff-4b77-9158-9ad2a0d23095 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:55.312055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 198/403] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:55 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:55.319601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768686a-32ff-4b77-9158-9ad2a0d23095 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:55.320509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 206/404] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:55.414054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:55.429911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a3c80-0620-4977-9d0e-29b7e68a1f54 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:55.483427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768686a-32ff-4b77-9158-9ad2a0d23095 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:55.571649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb29ed8-faba-401b-8b41-856ecbaccdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:55.572245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb29ed8-faba-401b-8b41-856ecbaccdca None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:55.572526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb29ed8-faba-401b-8b41-856ecbaccdca None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:55.572819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb29ed8-faba-401b-8b41-856ecbaccdca None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df/groups/8eede211d23743ed98fb3e4b7bf22efe/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:55.576014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:55.577049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb29ed8-faba-401b-8b41-856ecbaccdca None admin] RBAC: Authorizing `identity:create_grant(project_id=caedb3d1f7f94504824174418c72d3df, group_id=8eede211d23743ed98fb3e4b7bf22efe, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:55.578782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb29ed8-faba-401b-8b41-856ecbaccdca None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:55.583530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a3c80-0620-4977-9d0e-29b7e68a1f54 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:55.607097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 199/405] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Sep 15 10:42:55 2022] PUT /identity/v3/projects/caedb3d1f7f94504824174418c72d3df/groups/8eede211d23743ed98fb3e4b7bf22efe/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:55.641042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885c100-2131-45d8-80ee-04c5ae93d555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:55.642325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885c100-2131-45d8-80ee-04c5ae93d555 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:55.642609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885c100-2131-45d8-80ee-04c5ae93d555 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:55.642877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885c100-2131-45d8-80ee-04c5ae93d555 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:55.655322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb29ed8-faba-401b-8b41-856ecbaccdca None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:55.660640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885c100-2131-45d8-80ee-04c5ae93d555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b885c100-2131-45d8-80ee-04c5ae93d555 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:55.666143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:55.667093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:55.668316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 207/406] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:55.673550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885c100-2131-45d8-80ee-04c5ae93d555 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:55.851036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885c100-2131-45d8-80ee-04c5ae93d555 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:55.858074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb29ed8-faba-401b-8b41-856ecbaccdca None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:55.877236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:55.901853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ea8a0-479f-4f45-bf64-e32954c9d267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:55.902371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ea8a0-479f-4f45-bf64-e32954c9d267 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:55.902654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ea8a0-479f-4f45-bf64-e32954c9d267 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:55.902975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ea8a0-479f-4f45-bf64-e32954c9d267 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:55.915341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ea8a0-479f-4f45-bf64-e32954c9d267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:55.919003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e495517-9fb0-494e-9f95-c45617c5a5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:55.919875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e495517-9fb0-494e-9f95-c45617c5a5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:55.920358 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e495517-9fb0-494e-9f95-c45617c5a5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-ea5ea8a0-479f-4f45-bf64-e32954c9d267 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:55.920462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:55.921348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:55.922144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 200/407] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:55.924006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e495517-9fb0-494e-9f95-c45617c5a5c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:55.924706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e495517-9fb0-494e-9f95-c45617c5a5c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:55.925040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e495517-9fb0-494e-9f95-c45617c5a5c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:55.926431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e495517-9fb0-494e-9f95-c45617c5a5c1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:55.927441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ea8a0-479f-4f45-bf64-e32954c9d267 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:55.945034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 208/408] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:55.950251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e495517-9fb0-494e-9f95-c45617c5a5c1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:55.996548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:56.096146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ea8a0-479f-4f45-bf64-e32954c9d267 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:56.137845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e495517-9fb0-494e-9f95-c45617c5a5c1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:56.144606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bbcff-61b5-40ad-a124-a28bcf3dfa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:56.146535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bbcff-61b5-40ad-a124-a28bcf3dfa60 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.151207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bbcff-61b5-40ad-a124-a28bcf3dfa60 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.151711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bbcff-61b5-40ad-a124-a28bcf3dfa60 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.152657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bbcff-61b5-40ad-a124-a28bcf3dfa60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:56.153222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bbcff-61b5-40ad-a124-a28bcf3dfa60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:56.154835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bbcff-61b5-40ad-a124-a28bcf3dfa60 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:56.171687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 201/409] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:56.177724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bbcff-61b5-40ad-a124-a28bcf3dfa60 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:56.215166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3fee9-d84d-4f9b-82c5-d198fb581fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:56.216984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3fee9-d84d-4f9b-82c5-d198fb581fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.217274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3fee9-d84d-4f9b-82c5-d198fb581fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.217556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3fee9-d84d-4f9b-82c5-d198fb581fc5 None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.218745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3fee9-d84d-4f9b-82c5-d198fb581fc5 None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:56.220763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3fee9-d84d-4f9b-82c5-d198fb581fc5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:56.222743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 209/410] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:55 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:56.230548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3fee9-d84d-4f9b-82c5-d198fb581fc5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:56.246978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:56.366689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bbcff-61b5-40ad-a124-a28bcf3dfa60 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:56.388027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:56.405093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3fee9-d84d-4f9b-82c5-d198fb581fc5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:56.428236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28d66-4e0d-431f-ab04-5698c73a99d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:56.428902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28d66-4e0d-431f-ab04-5698c73a99d0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.429187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28d66-4e0d-431f-ab04-5698c73a99d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.429462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28d66-4e0d-431f-ab04-5698c73a99d0 None admin] PATH_INFO: `/v3/groups/c20709d488a44293b4d73fa0787c394b` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.431743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb28d66-4e0d-431f-ab04-5698c73a99d0 None admin] RBAC: Authorizing `identity:get_group(group_id=c20709d488a44293b4d73fa0787c394b)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:56.433227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb28d66-4e0d-431f-ab04-5698c73a99d0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:56.434844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 202/411] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:56 2022] GET /identity/v3/groups/c20709d488a44293b4d73fa0787c394b => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:56.441117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28d66-4e0d-431f-ab04-5698c73a99d0 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:56.472529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3060d-72b5-4b62-bbc9-d43ef6a7bc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:56.473291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3060d-72b5-4b62-bbc9-d43ef6a7bc87 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.473582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3060d-72b5-4b62-bbc9-d43ef6a7bc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.473858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3060d-72b5-4b62-bbc9-d43ef6a7bc87 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.476090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3060d-72b5-4b62-bbc9-d43ef6a7bc87 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:56.479379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3060d-72b5-4b62-bbc9-d43ef6a7bc87 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:56.485289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 210/412] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:56 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:56.486754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3060d-72b5-4b62-bbc9-d43ef6a7bc87 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:56.623014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3060d-72b5-4b62-bbc9-d43ef6a7bc87 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:56.635750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28d66-4e0d-431f-ab04-5698c73a99d0 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:56.638887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:56.665232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556345f6-b74d-45c4-b2fb-7af02b5f4804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:56.666212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556345f6-b74d-45c4-b2fb-7af02b5f4804 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.666775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556345f6-b74d-45c4-b2fb-7af02b5f4804 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.667199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556345f6-b74d-45c4-b2fb-7af02b5f4804 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df/groups/8eede211d23743ed98fb3e4b7bf22efe/roles/3518baf98ba54a28a9c79ec287469aff` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.669720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556345f6-b74d-45c4-b2fb-7af02b5f4804 None admin] RBAC: Authorizing `identity:create_grant(project_id=caedb3d1f7f94504824174418c72d3df, group_id=8eede211d23743ed98fb3e4b7bf22efe, role_id=3518baf98ba54a28a9c79ec287469aff)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:56.671541 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556345f6-b74d-45c4-b2fb-7af02b5f4804 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:56.679959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec265c-1930-4fb3-961a-825cc84976c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:56.680926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec265c-1930-4fb3-961a-825cc84976c4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.681316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec265c-1930-4fb3-961a-825cc84976c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.681693 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec265c-1930-4fb3-961a-825cc84976c4 None admin] PATH_INFO: `/v3/projects/70f07107ef09454996be197ead3ddf41` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.683655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec265c-1930-4fb3-961a-825cc84976c4 None admin] RBAC: Authorizing `identity:get_project(project_id=70f07107ef09454996be197ead3ddf41)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:56.685493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec265c-1930-4fb3-961a-825cc84976c4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:56.687743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 203/414] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:56 2022] GET /identity/v3/projects/70f07107ef09454996be197ead3ddf41 => generated 344 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:56.687917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 211/414] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Sep 15 10:42:56 2022] PUT /identity/v3/projects/caedb3d1f7f94504824174418c72d3df/groups/8eede211d23743ed98fb3e4b7bf22efe/roles/3518baf98ba54a28a9c79ec287469aff => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:42:56.692888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c98f7-f19c-482a-96a7-8c7e3ede55a7 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.692888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c98f7-f19c-482a-96a7-8c7e3ede55a7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.692888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c98f7-f19c-482a-96a7-8c7e3ede55a7 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.692888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 204/415] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:56.694398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556345f6-b74d-45c4-b2fb-7af02b5f4804 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:56.697587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed99fda-4220-4b47-8293-7e91d5267de7 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.698164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed99fda-4220-4b47-8293-7e91d5267de7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.698828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed99fda-4220-4b47-8293-7e91d5267de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.741140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:56.742918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-eed99fda-4220-4b47-8293-7e91d5267de7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:56.747127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-eed99fda-4220-4b47-8293-7e91d5267de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:56.893369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556345f6-b74d-45c4-b2fb-7af02b5f4804 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:56.902092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 205/416] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:56.920776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121a44-5686-4a88-85f4-7fc146f6ae41 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.921085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121a44-5686-4a88-85f4-7fc146f6ae41 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.921355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121a44-5686-4a88-85f4-7fc146f6ae41 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.922971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 206/417] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:56.928271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a3468-c4ec-49dc-bf0f-0b584b10b0d3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.929268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a3468-c4ec-49dc-bf0f-0b584b10b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.929552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a3468-c4ec-49dc-bf0f-0b584b10b0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.939584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3c09d-39d5-402a-b71f-1b871d1ddb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:56.940570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3c09d-39d5-402a-b71f-1b871d1ddb98 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:56.941113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3c09d-39d5-402a-b71f-1b871d1ddb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:56.941605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3c09d-39d5-402a-b71f-1b871d1ddb98 None admin] PATH_INFO: `/v3/projects/70f07107ef09454996be197ead3ddf41/groups/c20709d488a44293b4d73fa0787c394b/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:56.944198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3c09d-39d5-402a-b71f-1b871d1ddb98 None admin] RBAC: Authorizing `identity:create_grant(project_id=70f07107ef09454996be197ead3ddf41, group_id=c20709d488a44293b4d73fa0787c394b, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:56.945928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3c09d-39d5-402a-b71f-1b871d1ddb98 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:56.951651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ea3a3468-c4ec-49dc-bf0f-0b584b10b0d3 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:56.954550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ea3a3468-c4ec-49dc-bf0f-0b584b10b0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:56.960924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 212/418] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Sep 15 10:42:56 2022] PUT /identity/v3/projects/70f07107ef09454996be197ead3ddf41/groups/c20709d488a44293b4d73fa0787c394b/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:42:57.047097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 207/419] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:57.218953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3c09d-39d5-402a-b71f-1b871d1ddb98 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:57.314895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:57.337919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a3468-c4ec-49dc-bf0f-0b584b10b0d3 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:57.345813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3c09d-39d5-402a-b71f-1b871d1ddb98 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:57.375040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:57.406642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999ce61-5b29-4ea9-9b3c-9a2875d1af1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:57.407325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999ce61-5b29-4ea9-9b3c-9a2875d1af1a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:57.407615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999ce61-5b29-4ea9-9b3c-9a2875d1af1a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:57.407888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999ce61-5b29-4ea9-9b3c-9a2875d1af1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:57.431665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999ce61-5b29-4ea9-9b3c-9a2875d1af1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-a999ce61-5b29-4ea9-9b3c-9a2875d1af1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:57.445980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:57.446918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:57.447731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 213/420] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:57.453664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a3468-c4ec-49dc-bf0f-0b584b10b0d3 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:57.455928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999ce61-5b29-4ea9-9b3c-9a2875d1af1a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:57.493687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cb785-ca82-45bc-abae-22035acb4b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:57.494257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cb785-ca82-45bc-abae-22035acb4b99 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:57.494531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cb785-ca82-45bc-abae-22035acb4b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:57.494806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cb785-ca82-45bc-abae-22035acb4b99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:57.507330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cb785-ca82-45bc-abae-22035acb4b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-701cb785-ca82-45bc-abae-22035acb4b99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:57.512484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:57.514634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:57.517193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:57.521037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 208/421] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:57.521037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cb785-ca82-45bc-abae-22035acb4b99 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:57.633228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cb785-ca82-45bc-abae-22035acb4b99 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:57.645215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:57.674119 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a1f95-afa6-44e6-aee3-29870e5ad62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:57.674912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a1f95-afa6-44e6-aee3-29870e5ad62c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:57.675392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a1f95-afa6-44e6-aee3-29870e5ad62c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:57.675689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a1f95-afa6-44e6-aee3-29870e5ad62c None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:57.676376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a1f95-afa6-44e6-aee3-29870e5ad62c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:57.676696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a1f95-afa6-44e6-aee3-29870e5ad62c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:57.678141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a1f95-afa6-44e6-aee3-29870e5ad62c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:57.693083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 209/422] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:57.695566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a1f95-afa6-44e6-aee3-29870e5ad62c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:57.705146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999ce61-5b29-4ea9-9b3c-9a2875d1af1a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:57.806462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a1f95-afa6-44e6-aee3-29870e5ad62c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:57.843606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a38d52-b343-4961-9ba2-bac0ae69ecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:57.844224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a38d52-b343-4961-9ba2-bac0ae69ecf0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:57.845305 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a38d52-b343-4961-9ba2-bac0ae69ecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:57.845734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a38d52-b343-4961-9ba2-bac0ae69ecf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:57.846537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869aa7a-5613-4ef4-a4da-3eadebd501c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:57.847161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869aa7a-5613-4ef4-a4da-3eadebd501c6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:57.847442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869aa7a-5613-4ef4-a4da-3eadebd501c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:57.847716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869aa7a-5613-4ef4-a4da-3eadebd501c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:57.848391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869aa7a-5613-4ef4-a4da-3eadebd501c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:57.848691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869aa7a-5613-4ef4-a4da-3eadebd501c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:57.850113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869aa7a-5613-4ef4-a4da-3eadebd501c6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:57.863934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:57.877190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:57.878465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 214/423] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:57.886037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869aa7a-5613-4ef4-a4da-3eadebd501c6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:57.943276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a38d52-b343-4961-9ba2-bac0ae69ecf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:57.944759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a38d52-b343-4961-9ba2-bac0ae69ecf0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:58.075583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869aa7a-5613-4ef4-a4da-3eadebd501c6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:58.137650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 210/424] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 10:42:58.140142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385992a-76cd-4ee8-b7e7-1de64f6b8953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:58.140754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385992a-76cd-4ee8-b7e7-1de64f6b8953 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:58.141063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385992a-76cd-4ee8-b7e7-1de64f6b8953 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:58.141880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385992a-76cd-4ee8-b7e7-1de64f6b8953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:58.144982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a38d52-b343-4961-9ba2-bac0ae69ecf0 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:58.223906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:58.241486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:58.244371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385992a-76cd-4ee8-b7e7-1de64f6b8953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:58.246604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385992a-76cd-4ee8-b7e7-1de64f6b8953 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:58.285967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a38d52-b343-4961-9ba2-bac0ae69ecf0 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:58.328181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c5a3e-89f2-4b77-a1d2-61aede1bb321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:58.329599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c5a3e-89f2-4b77-a1d2-61aede1bb321 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:58.330044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c5a3e-89f2-4b77-a1d2-61aede1bb321 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:58.330601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c5a3e-89f2-4b77-a1d2-61aede1bb321 None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:58.332231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c5a3e-89f2-4b77-a1d2-61aede1bb321 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:58.334994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c5a3e-89f2-4b77-a1d2-61aede1bb321 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:58.341702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 211/425] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:58 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:58.343267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad45e83-7b30-4204-bf14-1068dc129073 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:58.343700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad45e83-7b30-4204-bf14-1068dc129073 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:58.344122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad45e83-7b30-4204-bf14-1068dc129073 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:58.346980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 212/426] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:42:58.348006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad45e83-7b30-4204-bf14-1068dc129073 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:58.382014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 215/427] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:42:58.384504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff47d32-2fd9-487e-85ce-de8408829d5b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:58.385032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff47d32-2fd9-487e-85ce-de8408829d5b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:58.385826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff47d32-2fd9-487e-85ce-de8408829d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:58.410229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6ff47d32-2fd9-487e-85ce-de8408829d5b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:58.412562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6ff47d32-2fd9-487e-85ce-de8408829d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:58.464733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad45e83-7b30-4204-bf14-1068dc129073 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:58.489588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:58.503811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64fe7c-cdb0-4be4-98bf-c3fce83b64d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:58.504451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64fe7c-cdb0-4be4-98bf-c3fce83b64d9 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:58.504732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64fe7c-cdb0-4be4-98bf-c3fce83b64d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:58.505026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64fe7c-cdb0-4be4-98bf-c3fce83b64d9 None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:58.506356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64fe7c-cdb0-4be4-98bf-c3fce83b64d9 None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:58.507583 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64fe7c-cdb0-4be4-98bf-c3fce83b64d9 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:58.509571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 213/428] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:58 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:58.510183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64fe7c-cdb0-4be4-98bf-c3fce83b64d9 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:58.510931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 216/429] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:42:58.518017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff47d32-2fd9-487e-85ce-de8408829d5b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:58.561649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:58.656339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64fe7c-cdb0-4be4-98bf-c3fce83b64d9 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:58.665657 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff47d32-2fd9-487e-85ce-de8408829d5b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:58.691658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d830d-f8d1-4491-9689-913fdb82874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:58.692308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d830d-f8d1-4491-9689-913fdb82874f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:58.692588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d830d-f8d1-4491-9689-913fdb82874f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:58.692888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d830d-f8d1-4491-9689-913fdb82874f None admin] PATH_INFO: `/v3/projects/a9cd5f1606ad462482bd725a3a3537a3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:58.694021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d830d-f8d1-4491-9689-913fdb82874f None admin] RBAC: Authorizing `identity:get_project(project_id=a9cd5f1606ad462482bd725a3a3537a3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:58.695464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d830d-f8d1-4491-9689-913fdb82874f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:58.697426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 214/430] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:58 2022] GET /identity/v3/projects/a9cd5f1606ad462482bd725a3a3537a3 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:58.702938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d830d-f8d1-4491-9689-913fdb82874f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:58.712925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9bc83-012a-4c6a-865b-a5cf1c91d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:58.713998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9bc83-012a-4c6a-865b-a5cf1c91d7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:58.714487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9bc83-012a-4c6a-865b-a5cf1c91d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:58.715057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9bc83-012a-4c6a-865b-a5cf1c91d7e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:58.716005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9bc83-012a-4c6a-865b-a5cf1c91d7e2 None admin] RBAC: Adding query filter params (group.id=8eede211d23743ed98fb3e4b7bf22efe, role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:58.716518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9bc83-012a-4c6a-865b-a5cf1c91d7e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:58.718386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9bc83-012a-4c6a-865b-a5cf1c91d7e2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:58.732003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 217/431] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:42:58 2022] GET /identity/v3/role_assignments?group.id=8eede211d23743ed98fb3e4b7bf22efe&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=3518baf98ba54a28a9c79ec287469aff => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:58.767244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:58.813362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d830d-f8d1-4491-9689-913fdb82874f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:58.851288 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb019d-61ee-453d-b4b1-72089d60c254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:58.851926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb019d-61ee-453d-b4b1-72089d60c254 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:58.852205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb019d-61ee-453d-b4b1-72089d60c254 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:58.852481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb019d-61ee-453d-b4b1-72089d60c254 None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:58.853703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb019d-61ee-453d-b4b1-72089d60c254 None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:58.855165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb019d-61ee-453d-b4b1-72089d60c254 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:58.857056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 215/432] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:58 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:58.863825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9bc83-012a-4c6a-865b-a5cf1c91d7e2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:58.910336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb019d-61ee-453d-b4b1-72089d60c254 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:59.027696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:59.038369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9bc83-012a-4c6a-865b-a5cf1c91d7e2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:59.072474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb019d-61ee-453d-b4b1-72089d60c254 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:59.084974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76526f4-c958-45da-bc92-6de206b1e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:59.086047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76526f4-c958-45da-bc92-6de206b1e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.086331 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76526f4-c958-45da-bc92-6de206b1e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.087005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76526f4-c958-45da-bc92-6de206b1e6c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.087753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76526f4-c958-45da-bc92-6de206b1e6c7 None admin] RBAC: Adding query filter params (group.id=8eede211d23743ed98fb3e4b7bf22efe, role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=a9cd5f1606ad462482bd725a3a3537a3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:59.088319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76526f4-c958-45da-bc92-6de206b1e6c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:59.090416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76526f4-c958-45da-bc92-6de206b1e6c7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:59.100067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 218/433] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:42:58 2022] GET /identity/v3/role_assignments?group.id=8eede211d23743ed98fb3e4b7bf22efe&scope.project.id=a9cd5f1606ad462482bd725a3a3537a3&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 611 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:42:59.115758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987110-4cc0-47f1-bebf-788ed0813649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:59.116432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987110-4cc0-47f1-bebf-788ed0813649 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.116765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987110-4cc0-47f1-bebf-788ed0813649 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.117072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987110-4cc0-47f1-bebf-788ed0813649 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.129344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d987110-4cc0-47f1-bebf-788ed0813649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-3d987110-4cc0-47f1-bebf-788ed0813649 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:59.134517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:59.135485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:42:59.136316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:59.137545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 216/434] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:42:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:42:59.143248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76526f4-c958-45da-bc92-6de206b1e6c7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:59.274863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76526f4-c958-45da-bc92-6de206b1e6c7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:59.316615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef9f90-e134-4a4f-b6b6-c7b5ddac0401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:59.317266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef9f90-e134-4a4f-b6b6-c7b5ddac0401 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.317550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef9f90-e134-4a4f-b6b6-c7b5ddac0401 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.317831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef9f90-e134-4a4f-b6b6-c7b5ddac0401 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.318500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef9f90-e134-4a4f-b6b6-c7b5ddac0401 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:42:59.318806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef9f90-e134-4a4f-b6b6-c7b5ddac0401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:59.320192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef9f90-e134-4a4f-b6b6-c7b5ddac0401 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:59.331455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 219/435] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:42:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:59.335726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987110-4cc0-47f1-bebf-788ed0813649 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:59.392601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e6e1d-eaf2-4aa5-81dc-e3817a4b4a59 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.392920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e6e1d-eaf2-4aa5-81dc-e3817a4b4a59 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.393176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e6e1d-eaf2-4aa5-81dc-e3817a4b4a59 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.394773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 220/436] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:42:59.399755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d48750-ff26-4eb2-94ef-c1a9356e0863 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.400041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d48750-ff26-4eb2-94ef-c1a9356e0863 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.400313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d48750-ff26-4eb2-94ef-c1a9356e0863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.421413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c0d48750-ff26-4eb2-94ef-c1a9356e0863 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:59.422601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c0d48750-ff26-4eb2-94ef-c1a9356e0863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:59.437669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:59.467684 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987110-4cc0-47f1-bebf-788ed0813649 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:59.512203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 221/437] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:59.512685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c5740-39be-450a-a40d-dabfc1098bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:59.513415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c5740-39be-450a-a40d-dabfc1098bca None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.513697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c5740-39be-450a-a40d-dabfc1098bca None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.513973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c5740-39be-450a-a40d-dabfc1098bca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.583996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:59.598576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c5740-39be-450a-a40d-dabfc1098bca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:59.600070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c5740-39be-450a-a40d-dabfc1098bca None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:59.639218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec921ed4-7dcc-4daf-8e6d-64e89b8da90c None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.639560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec921ed4-7dcc-4daf-8e6d-64e89b8da90c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.639774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec921ed4-7dcc-4daf-8e6d-64e89b8da90c None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.641905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 222/438] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:42:59.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5bfd8-9251-4aae-8422-77c47c357bee None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.647863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5bfd8-9251-4aae-8422-77c47c357bee None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.649768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5bfd8-9251-4aae-8422-77c47c357bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.672105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7ae5bfd8-9251-4aae-8422-77c47c357bee None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:42:59.673982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ae5bfd8-9251-4aae-8422-77c47c357bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:42:59.726283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 217/439] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:42:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 10:42:59.731313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c5740-39be-450a-a40d-dabfc1098bca None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:59.765745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 223/440] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:42:59.789470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5bfd8-9251-4aae-8422-77c47c357bee None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:59.838797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:59.858116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c5740-39be-450a-a40d-dabfc1098bca None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:59.896545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:42:59.898033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e054d4-12ce-402c-82b0-4560a7f743e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:59.898821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e054d4-12ce-402c-82b0-4560a7f743e2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.898930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e054d4-12ce-402c-82b0-4560a7f743e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.899469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e054d4-12ce-402c-82b0-4560a7f743e2 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.900609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e054d4-12ce-402c-82b0-4560a7f743e2 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:42:59.902113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e054d4-12ce-402c-82b0-4560a7f743e2 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:42:59.905481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 218/441] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:42:59 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:42:59.910986 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e054d4-12ce-402c-82b0-4560a7f743e2 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:42:59.923555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5bfd8-9251-4aae-8422-77c47c357bee None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:42:59.968625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38473a77-53d7-4663-997d-dd5d56965345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:42:59.969630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38473a77-53d7-4663-997d-dd5d56965345 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:42:59.970265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38473a77-53d7-4663-997d-dd5d56965345 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:42:59.970549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38473a77-53d7-4663-997d-dd5d56965345 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:42:59.983318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38473a77-53d7-4663-997d-dd5d56965345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-38473a77-53d7-4663-997d-dd5d56965345 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:42:59.988303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:42:59.989302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:42:59.990110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 224/442] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:42:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:42:59.995030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38473a77-53d7-4663-997d-dd5d56965345 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.020339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e054d4-12ce-402c-82b0-4560a7f743e2 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.056775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49242884-5826-4ecb-8eb9-b44b1c52a925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.057430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49242884-5826-4ecb-8eb9-b44b1c52a925 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.057705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49242884-5826-4ecb-8eb9-b44b1c52a925 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.057974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49242884-5826-4ecb-8eb9-b44b1c52a925 None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.059072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49242884-5826-4ecb-8eb9-b44b1c52a925 None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:00.060501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49242884-5826-4ecb-8eb9-b44b1c52a925 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:00.062079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 219/443] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:42:59 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:00.062825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49242884-5826-4ecb-8eb9-b44b1c52a925 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.088219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:00.098992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38473a77-53d7-4663-997d-dd5d56965345 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.132355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45665d-455b-4e78-9a22-f26aeb791ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.133002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45665d-455b-4e78-9a22-f26aeb791ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.133285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45665d-455b-4e78-9a22-f26aeb791ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.133558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45665d-455b-4e78-9a22-f26aeb791ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.134221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45665d-455b-4e78-9a22-f26aeb791ec7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:00.134526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45665d-455b-4e78-9a22-f26aeb791ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:00.135906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45665d-455b-4e78-9a22-f26aeb791ec7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:00.143817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 225/444] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:42:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:00.143817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45665d-455b-4e78-9a22-f26aeb791ec7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.147378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:00.168059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49242884-5826-4ecb-8eb9-b44b1c52a925 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.202361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1dbce-fae3-4a49-82c9-b098f9754554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.203253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1dbce-fae3-4a49-82c9-b098f9754554 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.203529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1dbce-fae3-4a49-82c9-b098f9754554 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.204415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1dbce-fae3-4a49-82c9-b098f9754554 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.214696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1dbce-fae3-4a49-82c9-b098f9754554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-f0a1dbce-fae3-4a49-82c9-b098f9754554 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:00.219718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:00.220635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 220/445] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:00.221745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1dbce-fae3-4a49-82c9-b098f9754554 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.247104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45665d-455b-4e78-9a22-f26aeb791ec7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.282054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837125f9-bc67-40ce-bec7-9ac94bb436be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.282854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837125f9-bc67-40ce-bec7-9ac94bb436be None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.283134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837125f9-bc67-40ce-bec7-9ac94bb436be None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.283404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837125f9-bc67-40ce-bec7-9ac94bb436be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.284117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837125f9-bc67-40ce-bec7-9ac94bb436be None admin] RBAC: Adding query filter params (group.id=8eede211d23743ed98fb3e4b7bf22efe, role.id=4cef964ca8d143eaadbb479db64250f1, scope.project.id=caedb3d1f7f94504824174418c72d3df) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:00.284420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837125f9-bc67-40ce-bec7-9ac94bb436be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:00.285875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837125f9-bc67-40ce-bec7-9ac94bb436be None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:00.291392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 226/446] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/role_assignments?group.id=8eede211d23743ed98fb3e4b7bf22efe&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=4cef964ca8d143eaadbb479db64250f1 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:00.292157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837125f9-bc67-40ce-bec7-9ac94bb436be None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.314520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:00.337324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1dbce-fae3-4a49-82c9-b098f9754554 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.369619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:00.379710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0742-14e8-42c8-a216-c5ce8dc59aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.380341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0742-14e8-42c8-a216-c5ce8dc59aaf None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.380625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0742-14e8-42c8-a216-c5ce8dc59aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.380935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0742-14e8-42c8-a216-c5ce8dc59aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.408962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837125f9-bc67-40ce-bec7-9ac94bb436be None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.445745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7236a-e9f5-403c-a44d-e29334fef82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.446392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7236a-e9f5-403c-a44d-e29334fef82a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.446676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7236a-e9f5-403c-a44d-e29334fef82a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.446947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7236a-e9f5-403c-a44d-e29334fef82a None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.447624 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7236a-e9f5-403c-a44d-e29334fef82a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:00.447928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7236a-e9f5-403c-a44d-e29334fef82a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:00.449384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7236a-e9f5-403c-a44d-e29334fef82a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:00.459075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 227/447] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:00.464823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7236a-e9f5-403c-a44d-e29334fef82a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.468685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d0742-14e8-42c8-a216-c5ce8dc59aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:00.470499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d0742-14e8-42c8-a216-c5ce8dc59aaf None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:00.567680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:00.584955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7236a-e9f5-403c-a44d-e29334fef82a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.596017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 221/448] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:00.601704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0742-14e8-42c8-a216-c5ce8dc59aaf None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.618523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:00.627663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad3642-2ad9-43c8-80a6-7b57ee4e0de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.628209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad3642-2ad9-43c8-80a6-7b57ee4e0de4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.628493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad3642-2ad9-43c8-80a6-7b57ee4e0de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.628767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad3642-2ad9-43c8-80a6-7b57ee4e0de4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.708084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad3642-2ad9-43c8-80a6-7b57ee4e0de4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:00.709595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad3642-2ad9-43c8-80a6-7b57ee4e0de4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:00.716971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0742-14e8-42c8-a216-c5ce8dc59aaf None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.749981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbebd5-01f1-4832-b8bf-738c43037c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.751137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbebd5-01f1-4832-b8bf-738c43037c38 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.751394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbebd5-01f1-4832-b8bf-738c43037c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.751662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbebd5-01f1-4832-b8bf-738c43037c38 None admin] PATH_INFO: `/v3/projects/caedb3d1f7f94504824174418c72d3df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.752777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbebd5-01f1-4832-b8bf-738c43037c38 None admin] RBAC: Authorizing `identity:get_project(project_id=caedb3d1f7f94504824174418c72d3df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:00.754246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbebd5-01f1-4832-b8bf-738c43037c38 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:00.755869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 222/449] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/projects/caedb3d1f7f94504824174418c72d3df => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:00.761236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbebd5-01f1-4832-b8bf-738c43037c38 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.807160 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 228/450] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 10:43:00.809882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:00.812868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad3642-2ad9-43c8-80a6-7b57ee4e0de4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.840487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:00.862324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbebd5-01f1-4832-b8bf-738c43037c38 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.895764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acbeaa-7bd2-4e97-b819-1714f8f8e884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.896314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acbeaa-7bd2-4e97-b819-1714f8f8e884 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.896588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acbeaa-7bd2-4e97-b819-1714f8f8e884 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.896887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acbeaa-7bd2-4e97-b819-1714f8f8e884 None admin] PATH_INFO: `/v3/groups/8eede211d23743ed98fb3e4b7bf22efe` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.897992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acbeaa-7bd2-4e97-b819-1714f8f8e884 None admin] RBAC: Authorizing `identity:get_group(group_id=8eede211d23743ed98fb3e4b7bf22efe)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:00.899437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acbeaa-7bd2-4e97-b819-1714f8f8e884 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:00.901013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 223/451] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/groups/8eede211d23743ed98fb3e4b7bf22efe => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:00.907175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acbeaa-7bd2-4e97-b819-1714f8f8e884 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:00.918088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad3642-2ad9-43c8-80a6-7b57ee4e0de4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:00.958453 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6926734-d359-45cc-81e7-3c5a4904b332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:00.959088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6926734-d359-45cc-81e7-3c5a4904b332 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:00.959369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6926734-d359-45cc-81e7-3c5a4904b332 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:00.959636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6926734-d359-45cc-81e7-3c5a4904b332 None admin] PATH_INFO: `/v3/projects/70f07107ef09454996be197ead3ddf41` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:00.960788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6926734-d359-45cc-81e7-3c5a4904b332 None admin] RBAC: Authorizing `identity:get_project(project_id=70f07107ef09454996be197ead3ddf41)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:00.962265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6926734-d359-45cc-81e7-3c5a4904b332 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:00.963886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 229/452] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/projects/70f07107ef09454996be197ead3ddf41 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:00.969819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6926734-d359-45cc-81e7-3c5a4904b332 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:01.008363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acbeaa-7bd2-4e97-b819-1714f8f8e884 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:01.025783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:01.043094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf00be-98c8-4db8-a66a-464150ebabf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:01.043689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf00be-98c8-4db8-a66a-464150ebabf4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:01.043963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf00be-98c8-4db8-a66a-464150ebabf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:01.044237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf00be-98c8-4db8-a66a-464150ebabf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:01.044978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf00be-98c8-4db8-a66a-464150ebabf4 None admin] RBAC: Adding query filter params (group.id=8eede211d23743ed98fb3e4b7bf22efe, role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=caedb3d1f7f94504824174418c72d3df) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:01.045283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf00be-98c8-4db8-a66a-464150ebabf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:01.046723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf00be-98c8-4db8-a66a-464150ebabf4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:01.052630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 224/453] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/role_assignments?group.id=8eede211d23743ed98fb3e4b7bf22efe&scope.project.id=caedb3d1f7f94504824174418c72d3df&role.id=3518baf98ba54a28a9c79ec287469aff => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:01.063617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:01.070288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6926734-d359-45cc-81e7-3c5a4904b332 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:01.107666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb811b8-934e-42fc-aa1e-400460b765f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:01.108325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb811b8-934e-42fc-aa1e-400460b765f7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:01.108608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb811b8-934e-42fc-aa1e-400460b765f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:01.108917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb811b8-934e-42fc-aa1e-400460b765f7 None admin] PATH_INFO: `/v3/groups/c20709d488a44293b4d73fa0787c394b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:01.110657 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb811b8-934e-42fc-aa1e-400460b765f7 None admin] RBAC: Authorizing `identity:get_group(group_id=c20709d488a44293b4d73fa0787c394b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:01.112159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb811b8-934e-42fc-aa1e-400460b765f7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:01.113891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 230/454] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Sep 15 10:43:00 2022] GET /identity/v3/groups/c20709d488a44293b4d73fa0787c394b => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:01.119967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf00be-98c8-4db8-a66a-464150ebabf4 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:01.228299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf00be-98c8-4db8-a66a-464150ebabf4 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:01.262976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb03202-c15d-4fdd-9c0b-944a661b375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:01.263583 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb03202-c15d-4fdd-9c0b-944a661b375f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:01.263867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb03202-c15d-4fdd-9c0b-944a661b375f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:01.264139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb03202-c15d-4fdd-9c0b-944a661b375f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:01.264970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb03202-c15d-4fdd-9c0b-944a661b375f None admin] RBAC: Adding query filter params (group.id=c20709d488a44293b4d73fa0787c394b, role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=70f07107ef09454996be197ead3ddf41) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:01.265281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb03202-c15d-4fdd-9c0b-944a661b375f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:01.266796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb03202-c15d-4fdd-9c0b-944a661b375f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:01.272656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 225/455] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Sep 15 10:43:01 2022] GET /identity/v3/role_assignments?group.id=c20709d488a44293b4d73fa0787c394b&scope.project.id=70f07107ef09454996be197ead3ddf41&role.id=0be9777002c641b4a47f78df647cc173 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:03.960562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73b53d-1176-4519-942a-cf800ccfe9b4 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:03.961105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73b53d-1176-4519-942a-cf800ccfe9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:03.961211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73b53d-1176-4519-942a-cf800ccfe9b4 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:03.962748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 231/456] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:03.967690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274ddc-e189-4263-b53e-1bea0aaaf387 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:03.968445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274ddc-e189-4263-b53e-1bea0aaaf387 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:03.969500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274ddc-e189-4263-b53e-1bea0aaaf387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:03.997892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-39274ddc-e189-4263-b53e-1bea0aaaf387 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:03.999429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-39274ddc-e189-4263-b53e-1bea0aaaf387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:04.057438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:04.097457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 226/457] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:04.165850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d960b-de62-4826-b584-61cf531bf536 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:04.166187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d960b-de62-4826-b584-61cf531bf536 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:04.166410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d960b-de62-4826-b584-61cf531bf536 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:04.168050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 232/458] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:04.173538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38370686-bf37-4375-9773-35047d9a030f None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:04.173828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38370686-bf37-4375-9773-35047d9a030f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:04.174110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38370686-bf37-4375-9773-35047d9a030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:04.205819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-38370686-bf37-4375-9773-35047d9a030f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:04.207023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-38370686-bf37-4375-9773-35047d9a030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:04.298736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 227/459] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:43:04.387072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe83cca-c4cf-48c5-9402-41ed1b2df43e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:04.387704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe83cca-c4cf-48c5-9402-41ed1b2df43e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:04.388209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe83cca-c4cf-48c5-9402-41ed1b2df43e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:04.389821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 233/460] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:04.395243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d5ce3-9858-46a5-bf0c-931c8a5d28b9 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:04.395753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d5ce3-9858-46a5-bf0c-931c8a5d28b9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:04.396293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d5ce3-9858-46a5-bf0c-931c8a5d28b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:04.425804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3a7d5ce3-9858-46a5-bf0c-931c8a5d28b9 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:04.427330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3a7d5ce3-9858-46a5-bf0c-931c8a5d28b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:04.503217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe83cca-c4cf-48c5-9402-41ed1b2df43e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:04.515664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 228/461] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:04.681483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe83cca-c4cf-48c5-9402-41ed1b2df43e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:04.728678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10724dc-9be5-4c32-b85b-b88498aa7117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:04.729339 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10724dc-9be5-4c32-b85b-b88498aa7117 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:04.729622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10724dc-9be5-4c32-b85b-b88498aa7117 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:04.729895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10724dc-9be5-4c32-b85b-b88498aa7117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:04.740053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d5ce3-9858-46a5-bf0c-931c8a5d28b9 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:04.746468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10724dc-9be5-4c32-b85b-b88498aa7117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:04.747870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10724dc-9be5-4c32-b85b-b88498aa7117 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c10724dc-9be5-4c32-b85b-b88498aa7117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:04.770601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:04.771598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:04.772912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:04.772912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:04.772912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:04.772912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 234/462] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:04.780481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10724dc-9be5-4c32-b85b-b88498aa7117 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:04.804151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:04.807253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:04.852471 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d5ce3-9858-46a5-bf0c-931c8a5d28b9 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:04.888601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc964bc7-eff9-4208-997d-c55a1708eace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:04.889290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc964bc7-eff9-4208-997d-c55a1708eace None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:04.889574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc964bc7-eff9-4208-997d-c55a1708eace None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:04.889854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc964bc7-eff9-4208-997d-c55a1708eace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:04.894251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10724dc-9be5-4c32-b85b-b88498aa7117 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:04.902953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc964bc7-eff9-4208-997d-c55a1708eace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:04.905299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc964bc7-eff9-4208-997d-c55a1708eace None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-fc964bc7-eff9-4208-997d-c55a1708eace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:04.920823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:04.921779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:04.923296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:04.923296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:04.923296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:04.923296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 229/463] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:04.923816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932010c-1f49-4d4c-a7ab-90b2648cfbdb None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:04.923913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932010c-1f49-4d4c-a7ab-90b2648cfbdb None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:04.924156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932010c-1f49-4d4c-a7ab-90b2648cfbdb None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:04.925552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 230/464] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:04.926114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932010c-1f49-4d4c-a7ab-90b2648cfbdb None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:04.939222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9eb81-ac42-4c03-88b0-13d59dcf91a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:04.939995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9eb81-ac42-4c03-88b0-13d59dcf91a1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:04.940279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9eb81-ac42-4c03-88b0-13d59dcf91a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:04.940553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9eb81-ac42-4c03-88b0-13d59dcf91a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:04.941255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9eb81-ac42-4c03-88b0-13d59dcf91a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:04.941638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9eb81-ac42-4c03-88b0-13d59dcf91a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:04.943050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9eb81-ac42-4c03-88b0-13d59dcf91a1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:04.957311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 235/465] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:04.957984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9eb81-ac42-4c03-88b0-13d59dcf91a1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:05.023323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:05.040164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932010c-1f49-4d4c-a7ab-90b2648cfbdb None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:05.043567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:05.070204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9eb81-ac42-4c03-88b0-13d59dcf91a1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:05.079664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910a775-5997-4972-ad06-e0f8613362e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:05.080311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910a775-5997-4972-ad06-e0f8613362e3 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.080590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910a775-5997-4972-ad06-e0f8613362e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.080896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910a775-5997-4972-ad06-e0f8613362e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.096212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910a775-5997-4972-ad06-e0f8613362e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:05.097741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910a775-5997-4972-ad06-e0f8613362e3 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:05.108670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2fcb-756c-4502-8e0d-df953a2636ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:05.110139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2fcb-756c-4502-8e0d-df953a2636ce None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.110604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2fcb-756c-4502-8e0d-df953a2636ce None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.111140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2fcb-756c-4502-8e0d-df953a2636ce None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.111984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef2fcb-756c-4502-8e0d-df953a2636ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:05.112466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef2fcb-756c-4502-8e0d-df953a2636ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-9910a775-5997-4972-ad06-e0f8613362e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:05.113467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:05.114355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:05.115209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:05.115209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:05.115209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:05.115209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 231/466] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:05.115711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed1cfc-20a0-4dc3-b799-2866d5a9ea3c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.115975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed1cfc-20a0-4dc3-b799-2866d5a9ea3c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.116247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed1cfc-20a0-4dc3-b799-2866d5a9ea3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.116839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef2fcb-756c-4502-8e0d-df953a2636ce None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:05.135228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 236/467] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:05.136498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2fcb-756c-4502-8e0d-df953a2636ce None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:05.137625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-93ed1cfc-20a0-4dc3-b799-2866d5a9ea3c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:05.138821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-93ed1cfc-20a0-4dc3-b799-2866d5a9ea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:05.213403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 232/468] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:05.214256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed1cfc-20a0-4dc3-b799-2866d5a9ea3c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:05.260772 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2fcb-756c-4502-8e0d-df953a2636ce None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:05.293350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:05.300913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57561697-efa5-41f9-bbea-c7e6c1ad44fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:05.301574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57561697-efa5-41f9-bbea-c7e6c1ad44fd None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.301863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57561697-efa5-41f9-bbea-c7e6c1ad44fd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.302139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57561697-efa5-41f9-bbea-c7e6c1ad44fd None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.302904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57561697-efa5-41f9-bbea-c7e6c1ad44fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:05.304393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57561697-efa5-41f9-bbea-c7e6c1ad44fd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:05.314320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:05.335928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed1cfc-20a0-4dc3-b799-2866d5a9ea3c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:05.336329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 237/469] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:43:05 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:05.342984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57561697-efa5-41f9-bbea-c7e6c1ad44fd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:05.417236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71317486-a031-42d5-86fa-677cd1ab5a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:05.418213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71317486-a031-42d5-86fa-677cd1ab5a92 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.419656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71317486-a031-42d5-86fa-677cd1ab5a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.420131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71317486-a031-42d5-86fa-677cd1ab5a92 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.420999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71317486-a031-42d5-86fa-677cd1ab5a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:05.422869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71317486-a031-42d5-86fa-677cd1ab5a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:05.424640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71317486-a031-42d5-86fa-677cd1ab5a92 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:05.450612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 233/470] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:05.452220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e237d-38d0-4404-8e0f-ce90031fc2e2 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.452949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e237d-38d0-4404-8e0f-ce90031fc2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.453429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e237d-38d0-4404-8e0f-ce90031fc2e2 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.454925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 234/471] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:05.463872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9fc20-a299-4eaa-919e-0d17461a03c4 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.463872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9fc20-a299-4eaa-919e-0d17461a03c4 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.463872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9fc20-a299-4eaa-919e-0d17461a03c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.484392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-37c9fc20-a299-4eaa-919e-0d17461a03c4 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:05.486507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-37c9fc20-a299-4eaa-919e-0d17461a03c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:05.515374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57561697-efa5-41f9-bbea-c7e6c1ad44fd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:05.556522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d99f35-5d8b-4c19-b053-2c2412f8a02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:05.556991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99f35-5d8b-4c19-b053-2c2412f8a02e None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.557276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99f35-5d8b-4c19-b053-2c2412f8a02e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.557552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99f35-5d8b-4c19-b053-2c2412f8a02e None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.558530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d99f35-5d8b-4c19-b053-2c2412f8a02e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:05.559647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d99f35-5d8b-4c19-b053-2c2412f8a02e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:05.579695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 235/472] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:05.580470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9fc20-a299-4eaa-919e-0d17461a03c4 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:05.585154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 238/473] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:43:05 2022] POST /identity/v3/users => generated 248 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:05.585772 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d99f35-5d8b-4c19-b053-2c2412f8a02e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:05.649181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:05.654156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:05.709736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9fc20-a299-4eaa-919e-0d17461a03c4 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:05.713266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d99f35-5d8b-4c19-b053-2c2412f8a02e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:05.760344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cba51c-c601-41f8-9e02-2ab6654610b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:05.761113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cba51c-c601-41f8-9e02-2ab6654610b0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.761221 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e6caf-d600-423e-8255-f05e00e2642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:05.761391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cba51c-c601-41f8-9e02-2ab6654610b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.761676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cba51c-c601-41f8-9e02-2ab6654610b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.762042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e6caf-d600-423e-8255-f05e00e2642d None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.762392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e6caf-d600-423e-8255-f05e00e2642d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.762739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e6caf-d600-423e-8255-f05e00e2642d None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.763482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e6caf-d600-423e-8255-f05e00e2642d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:05.765082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e6caf-d600-423e-8255-f05e00e2642d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:05.774181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cba51c-c601-41f8-9e02-2ab6654610b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:05.776198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cba51c-c601-41f8-9e02-2ab6654610b0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:05.791550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 236/474] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:43:05 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-65cba51c-c601-41f8-9e02-2ab6654610b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:05.795757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:05.796700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:05.798523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:05.798523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:05.798523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:05.798523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 239/475] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:05.806238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e6caf-d600-423e-8255-f05e00e2642d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:05.926080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e6caf-d600-423e-8255-f05e00e2642d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:05.945558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:05.963212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cba51c-c601-41f8-9e02-2ab6654610b0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:05.968323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b656096-0660-4093-a814-03e786b58a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:05.970060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b656096-0660-4093-a814-03e786b58a25 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:05.970555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b656096-0660-4093-a814-03e786b58a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:05.971067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b656096-0660-4093-a814-03e786b58a25 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:05.971945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b656096-0660-4093-a814-03e786b58a25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:05.972487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b656096-0660-4093-a814-03e786b58a25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:05.974195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b656096-0660-4093-a814-03e786b58a25 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:05.992246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 237/476] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:05.998321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b656096-0660-4093-a814-03e786b58a25 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:06.102562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:06.105940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cba51c-c601-41f8-9e02-2ab6654610b0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:06.128791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b656096-0660-4093-a814-03e786b58a25 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:06.146534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:06.147148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:06.147430 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:06.147697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:06.148374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:06.149921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:06.159336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.sql.core [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 10:43:06.159336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 10:43:06.159336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [parameters: {'id': 'a19eeaa2cba74db0b01105ff8401f3ec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 10:43:06.159336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81387) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 10:43:06.169781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96c0d5-970c-4b83-861b-7e33dedcec39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:06.170452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96c0d5-970c-4b83-861b-7e33dedcec39 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:06.170737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96c0d5-970c-4b83-861b-7e33dedcec39 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:06.171060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96c0d5-970c-4b83-861b-7e33dedcec39 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:06.171709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96c0d5-970c-4b83-861b-7e33dedcec39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:06.173904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96c0d5-970c-4b83-861b-7e33dedcec39 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application conn.query(q) Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result.read() Sep 15 10:43:06.176998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return ref.to_dict() Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 10:43:06.177873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application next(self.gen) Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application yield resource Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application next(self.gen) Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise self.value Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application session.commit() Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 10:43:06.178822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.session.flush() Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application compat.raise_( Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 10:43:06.179727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 10:43:06.180616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application conn.query(q) Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result.read() Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 10:43:06.181474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [parameters: {'id': 'a19eeaa2cba74db0b01105ff8401f3ec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:06.182433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 10:43:06.183266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:06.184122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 240/477] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Sep 15 10:43:05 2022] POST /identity/v3/roles => generated 144 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:06.189616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:06.228257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 238/478] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:43:05 2022] POST /identity/v3/users => generated 249 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:43:06.323101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2b343-e06f-4995-bb9b-bc0efbb794bb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:06.363503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d06c49-f335-40fc-bb12-55700ce71b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:06.364166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d06c49-f335-40fc-bb12-55700ce71b71 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:06.364452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d06c49-f335-40fc-bb12-55700ce71b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:06.364730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d06c49-f335-40fc-bb12-55700ce71b71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:06.377211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d06c49-f335-40fc-bb12-55700ce71b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-34d06c49-f335-40fc-bb12-55700ce71b71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:06.382753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:06.383920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:06.384965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 241/479] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:06.390378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96c0d5-970c-4b83-861b-7e33dedcec39 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:06.438793 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:06.535986 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96c0d5-970c-4b83-861b-7e33dedcec39 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:06.585377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da17246-4ab4-43d9-b872-8b6ed09fccc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:06.586378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da17246-4ab4-43d9-b872-8b6ed09fccc8 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:06.586896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da17246-4ab4-43d9-b872-8b6ed09fccc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:06.587368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da17246-4ab4-43d9-b872-8b6ed09fccc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:06.588233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da17246-4ab4-43d9-b872-8b6ed09fccc8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:06.588761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da17246-4ab4-43d9-b872-8b6ed09fccc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:06.590360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da17246-4ab4-43d9-b872-8b6ed09fccc8 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:06.602563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 239/480] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:08.241313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a2718-b0eb-4910-8040-973cdddae859 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:08.241833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a2718-b0eb-4910-8040-973cdddae859 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:08.241944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a2718-b0eb-4910-8040-973cdddae859 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:08.243487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 242/481] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:08.248973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07609c-6f52-4354-949c-17242a9d631a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:08.249386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07609c-6f52-4354-949c-17242a9d631a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:08.249555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07609c-6f52-4354-949c-17242a9d631a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:08.272527 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-6c07609c-6f52-4354-949c-17242a9d631a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:08.273760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6c07609c-6f52-4354-949c-17242a9d631a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:08.360652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 240/482] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:08.436865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120ff6a-50b4-4640-940a-36e369001b3a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:08.437156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120ff6a-50b4-4640-940a-36e369001b3a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:08.437492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120ff6a-50b4-4640-940a-36e369001b3a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:08.439021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 243/483] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:08.444281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1127ac-c4a4-4bb3-93ab-5bfdd9223bb9 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:08.444573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1127ac-c4a4-4bb3-93ab-5bfdd9223bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:08.444874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1127ac-c4a4-4bb3-93ab-5bfdd9223bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:08.463500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-cf1127ac-c4a4-4bb3-93ab-5bfdd9223bb9 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:08.464681 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cf1127ac-c4a4-4bb3-93ab-5bfdd9223bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:08.483519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:08.551240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 241/484] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:08.698243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2414bd-9bae-41d1-b63f-80d86ad77643 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:08.698530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2414bd-9bae-41d1-b63f-80d86ad77643 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:08.699069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2414bd-9bae-41d1-b63f-80d86ad77643 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:08.701058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 244/485] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:08.702122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1127ac-c4a4-4bb3-93ab-5bfdd9223bb9 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:08.706159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244c7cd-cc86-4ce6-8343-718461a0e16d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:08.706866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244c7cd-cc86-4ce6-8343-718461a0e16d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:08.707370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244c7cd-cc86-4ce6-8343-718461a0e16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:08.726154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8244c7cd-cc86-4ce6-8343-718461a0e16d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:08.727783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8244c7cd-cc86-4ce6-8343-718461a0e16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:08.753493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:08.846260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 245/486] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:43:08.872100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1127ac-c4a4-4bb3-93ab-5bfdd9223bb9 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:08.907920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f559b5-55e5-417e-bdd7-8457e4c99b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:08.908555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f559b5-55e5-417e-bdd7-8457e4c99b93 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:08.908870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f559b5-55e5-417e-bdd7-8457e4c99b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:08.909151 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f559b5-55e5-417e-bdd7-8457e4c99b93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:08.919727 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f559b5-55e5-417e-bdd7-8457e4c99b93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-24f559b5-55e5-417e-bdd7-8457e4c99b93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:08.925257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:08.926133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:08.926953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 242/487] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:08.938020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244c7cd-cc86-4ce6-8343-718461a0e16d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:08.947284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f559b5-55e5-417e-bdd7-8457e4c99b93 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:08.965368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:09.131474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f559b5-55e5-417e-bdd7-8457e4c99b93 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:09.140104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244c7cd-cc86-4ce6-8343-718461a0e16d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:09.192701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d5bb6-1c51-485f-ad5e-736f40159e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:09.193374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d5bb6-1c51-485f-ad5e-736f40159e19 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.193657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d5bb6-1c51-485f-ad5e-736f40159e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.193978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d5bb6-1c51-485f-ad5e-736f40159e19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.194343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30d09f-18d9-4970-86c1-88c5daf1ae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:09.195226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30d09f-18d9-4970-86c1-88c5daf1ae47 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.196199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30d09f-18d9-4970-86c1-88c5daf1ae47 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.196503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30d09f-18d9-4970-86c1-88c5daf1ae47 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.197347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30d09f-18d9-4970-86c1-88c5daf1ae47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:09.197890 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30d09f-18d9-4970-86c1-88c5daf1ae47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:09.199294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30d09f-18d9-4970-86c1-88c5daf1ae47 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:09.209023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d5bb6-1c51-485f-ad5e-736f40159e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:09.214646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 246/488] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:09.214646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30d09f-18d9-4970-86c1-88c5daf1ae47 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-731d5bb6-1c51-485f-ad5e-736f40159e19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:09.214840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:09.215707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:09.216515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:09.217193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 243/489] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:09.217919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af36e5c-fea9-424e-9ad8-3c1b0310072c None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.218089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af36e5c-fea9-424e-9ad8-3c1b0310072c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.221708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af36e5c-fea9-424e-9ad8-3c1b0310072c None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.221708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 244/490] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:09.223230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af36e5c-fea9-424e-9ad8-3c1b0310072c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:09.233384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:09.308846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:09.326108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30d09f-18d9-4970-86c1-88c5daf1ae47 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:09.337418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af36e5c-fea9-424e-9ad8-3c1b0310072c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:09.365462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570cb1e-e5a9-4f6d-bd06-1d7620b0deb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:09.366128 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570cb1e-e5a9-4f6d-bd06-1d7620b0deb4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.366412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570cb1e-e5a9-4f6d-bd06-1d7620b0deb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.366690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570cb1e-e5a9-4f6d-bd06-1d7620b0deb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.378953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570cb1e-e5a9-4f6d-bd06-1d7620b0deb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:09.379152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca85894-307b-492a-85d6-232435ba26b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:09.379673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca85894-307b-492a-85d6-232435ba26b4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.379959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca85894-307b-492a-85d6-232435ba26b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.380233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca85894-307b-492a-85d6-232435ba26b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6570cb1e-e5a9-4f6d-bd06-1d7620b0deb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:09.384428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:09.385362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 247/491] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:09.390009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570cb1e-e5a9-4f6d-bd06-1d7620b0deb4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:09.395795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca85894-307b-492a-85d6-232435ba26b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:09.397315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca85894-307b-492a-85d6-232435ba26b4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-5ca85894-307b-492a-85d6-232435ba26b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:09.413921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:09.414904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:09.416350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:09.416350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:09.416350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:09.416350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 245/492] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:09.418048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd702b63-b2de-4468-8268-0e2a5c5ae86c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.418333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd702b63-b2de-4468-8268-0e2a5c5ae86c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.418607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd702b63-b2de-4468-8268-0e2a5c5ae86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.437290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-cd702b63-b2de-4468-8268-0e2a5c5ae86c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:09.438645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cd702b63-b2de-4468-8268-0e2a5c5ae86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:09.479344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:09.502591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570cb1e-e5a9-4f6d-bd06-1d7620b0deb4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:09.506808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 246/493] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:09.507655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd702b63-b2de-4468-8268-0e2a5c5ae86c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:09.543092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780d69e-a8a5-40b3-8db5-2c3d94f91e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:09.543707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780d69e-a8a5-40b3-8db5-2c3d94f91e1b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.543991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780d69e-a8a5-40b3-8db5-2c3d94f91e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.544268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780d69e-a8a5-40b3-8db5-2c3d94f91e1b None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.544994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780d69e-a8a5-40b3-8db5-2c3d94f91e1b None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:09.545305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780d69e-a8a5-40b3-8db5-2c3d94f91e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:09.546695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780d69e-a8a5-40b3-8db5-2c3d94f91e1b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:09.559620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 248/494] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:09.559620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780d69e-a8a5-40b3-8db5-2c3d94f91e1b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:09.579934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:09.627483 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd702b63-b2de-4468-8268-0e2a5c5ae86c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:09.664738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780d69e-a8a5-40b3-8db5-2c3d94f91e1b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:09.664965 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561dbef-01af-4039-b080-12a6c35a98b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:09.665936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561dbef-01af-4039-b080-12a6c35a98b0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.666474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561dbef-01af-4039-b080-12a6c35a98b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.667047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561dbef-01af-4039-b080-12a6c35a98b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.667708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561dbef-01af-4039-b080-12a6c35a98b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:09.668010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561dbef-01af-4039-b080-12a6c35a98b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:09.669409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561dbef-01af-4039-b080-12a6c35a98b0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:09.679051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 247/495] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:09.680215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aac8c1-9141-48d8-996e-a861e04571bd None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.680497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aac8c1-9141-48d8-996e-a861e04571bd None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.680769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aac8c1-9141-48d8-996e-a861e04571bd None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.685138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 248/496] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:09.685138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aac8c1-9141-48d8-996e-a861e04571bd None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:09.706676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484a0ea-0df9-4366-b588-f6189996f686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:09.707287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484a0ea-0df9-4366-b588-f6189996f686 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.707570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484a0ea-0df9-4366-b588-f6189996f686 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.707844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484a0ea-0df9-4366-b588-f6189996f686 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.708482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484a0ea-0df9-4366-b588-f6189996f686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:09.708783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484a0ea-0df9-4366-b588-f6189996f686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:09.710255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484a0ea-0df9-4366-b588-f6189996f686 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:09.717588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:09.724356 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 249/497] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:09.725255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484a0ea-0df9-4366-b588-f6189996f686 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:09.802950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aac8c1-9141-48d8-996e-a861e04571bd None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:09.820012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:09.834110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484a0ea-0df9-4366-b588-f6189996f686 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:09.838201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7a17b-1563-4361-8c17-ee7244d5e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:09.838828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7a17b-1563-4361-8c17-ee7244d5e639 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.839110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7a17b-1563-4361-8c17-ee7244d5e639 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.839377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7a17b-1563-4361-8c17-ee7244d5e639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.850247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7a17b-1563-4361-8c17-ee7244d5e639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:09.851704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7a17b-1563-4361-8c17-ee7244d5e639 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-c4f7a17b-1563-4361-8c17-ee7244d5e639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:09.865448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:09.866485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:09.867327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:09.867327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:09.867327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:09.867327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 249/498] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:09.867908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f57bd-df17-4686-af2c-0b54e5375771 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.868148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f57bd-df17-4686-af2c-0b54e5375771 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.868426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f57bd-df17-4686-af2c-0b54e5375771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.868517 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240f978-1599-4c27-a133-1fd40992e2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:09.869138 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240f978-1599-4c27-a133-1fd40992e2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:09.869419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240f978-1599-4c27-a133-1fd40992e2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:09.869691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240f978-1599-4c27-a133-1fd40992e2e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:09.880659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240f978-1599-4c27-a133-1fd40992e2e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:09.882236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240f978-1599-4c27-a133-1fd40992e2e8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:09.886918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-861f57bd-df17-4686-af2c-0b54e5375771 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:09.888107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-861f57bd-df17-4686-af2c-0b54e5375771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c240f978-1599-4c27-a133-1fd40992e2e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:09.897512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:09.898604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:09.899675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:09.899675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:09.899675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:09.899675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 250/499] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:09.899675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240f978-1599-4c27-a133-1fd40992e2e8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:09.949318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:09.960370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 250/500] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:09.961326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f57bd-df17-4686-af2c-0b54e5375771 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.023952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240f978-1599-4c27-a133-1fd40992e2e8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.060497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ee35a-dc30-4b89-a640-8fe34b2fd59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.060688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:10.061371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ee35a-dc30-4b89-a640-8fe34b2fd59f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.062010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ee35a-dc30-4b89-a640-8fe34b2fd59f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.063085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ee35a-dc30-4b89-a640-8fe34b2fd59f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.064521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f57bd-df17-4686-af2c-0b54e5375771 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.071878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ee35a-dc30-4b89-a640-8fe34b2fd59f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.073814 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ee35a-dc30-4b89-a640-8fe34b2fd59f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-235ee35a-dc30-4b89-a640-8fe34b2fd59f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:10.088102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:10.089301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:10.090445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:10.094074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 251/501] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Sep 15 10:43:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:10.094779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ee35a-dc30-4b89-a640-8fe34b2fd59f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.105110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa2146-9ea0-43ad-a38c-a938ed66868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.105992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa2146-9ea0-43ad-a38c-a938ed66868e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.106412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa2146-9ea0-43ad-a38c-a938ed66868e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.106832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa2146-9ea0-43ad-a38c-a938ed66868e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.118249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa2146-9ea0-43ad-a38c-a938ed66868e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-5bfa2146-9ea0-43ad-a38c-a938ed66868e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:10.123477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:10.124469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:10.126205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:10.128137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 251/502] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:10.129080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa2146-9ea0-43ad-a38c-a938ed66868e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.206486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ee35a-dc30-4b89-a640-8fe34b2fd59f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.216976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:10.239095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa2146-9ea0-43ad-a38c-a938ed66868e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.242456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae2b1a-19ec-424e-a57e-aac55314c5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.243062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae2b1a-19ec-424e-a57e-aac55314c5ec None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.243343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae2b1a-19ec-424e-a57e-aac55314c5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.243617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae2b1a-19ec-424e-a57e-aac55314c5ec None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.244247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae2b1a-19ec-424e-a57e-aac55314c5ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:10.244548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae2b1a-19ec-424e-a57e-aac55314c5ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.245984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae2b1a-19ec-424e-a57e-aac55314c5ec None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.257742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 252/503] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:10.258528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae2b1a-19ec-424e-a57e-aac55314c5ec None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.273386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067c419-773e-43fa-a5b7-499881deeddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.273991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067c419-773e-43fa-a5b7-499881deeddf None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.274268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067c419-773e-43fa-a5b7-499881deeddf None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.274536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067c419-773e-43fa-a5b7-499881deeddf None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.275171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067c419-773e-43fa-a5b7-499881deeddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:10.275472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067c419-773e-43fa-a5b7-499881deeddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.276859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067c419-773e-43fa-a5b7-499881deeddf None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.287973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 252/504] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:10.288955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067c419-773e-43fa-a5b7-499881deeddf None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.310279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:10.362387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae2b1a-19ec-424e-a57e-aac55314c5ec None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.388605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067c419-773e-43fa-a5b7-499881deeddf None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.395681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d69f65-3393-4b4c-a6f5-1a60f081621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.396282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69f65-3393-4b4c-a6f5-1a60f081621a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.396597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69f65-3393-4b4c-a6f5-1a60f081621a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.396905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69f65-3393-4b4c-a6f5-1a60f081621a None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.397634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d69f65-3393-4b4c-a6f5-1a60f081621a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:10.397922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d69f65-3393-4b4c-a6f5-1a60f081621a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.399355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d69f65-3393-4b4c-a6f5-1a60f081621a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.422819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101941e-a8b0-4124-9761-ae00b6076a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.423485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101941e-a8b0-4124-9761-ae00b6076a02 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.423764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101941e-a8b0-4124-9761-ae00b6076a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.424032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101941e-a8b0-4124-9761-ae00b6076a02 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.424682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101941e-a8b0-4124-9761-ae00b6076a02 None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:10.425019 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101941e-a8b0-4124-9761-ae00b6076a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.426379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101941e-a8b0-4124-9761-ae00b6076a02 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.434168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 253/505] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:10.434946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101941e-a8b0-4124-9761-ae00b6076a02 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.440123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 253/506] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:10.440123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d69f65-3393-4b4c-a6f5-1a60f081621a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.466216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:10.520458 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:10.537487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d69f65-3393-4b4c-a6f5-1a60f081621a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.543029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101941e-a8b0-4124-9761-ae00b6076a02 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.573915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14715dc-ddc0-413c-ab48-b8266649c039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.574652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14715dc-ddc0-413c-ab48-b8266649c039 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.574816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14715dc-ddc0-413c-ab48-b8266649c039 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.575086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14715dc-ddc0-413c-ab48-b8266649c039 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.580355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08fa73-989f-4c99-86ea-8c3156e6b09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.581007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08fa73-989f-4c99-86ea-8c3156e6b09d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.581286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08fa73-989f-4c99-86ea-8c3156e6b09d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.581554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08fa73-989f-4c99-86ea-8c3156e6b09d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.582383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14715dc-ddc0-413c-ab48-b8266649c039 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.583835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14715dc-ddc0-413c-ab48-b8266649c039 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.591914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08fa73-989f-4c99-86ea-8c3156e6b09d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.593375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08fa73-989f-4c99-86ea-8c3156e6b09d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c14715dc-ddc0-413c-ab48-b8266649c039 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:10.595748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:10.596842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 254/507] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:10.598002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14715dc-ddc0-413c-ab48-b8266649c039 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-ef08fa73-989f-4c99-86ea-8c3156e6b09d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:10.611573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:10.612635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:10.617824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:10.617824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:10.617824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:10.617824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 254/508] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:10.617824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08fa73-989f-4c99-86ea-8c3156e6b09d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.688746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:10.701358 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14715dc-ddc0-413c-ab48-b8266649c039 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.712443 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08fa73-989f-4c99-86ea-8c3156e6b09d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.736250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add976e4-548f-4037-9332-9d6285b795b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.736846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add976e4-548f-4037-9332-9d6285b795b8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.737135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add976e4-548f-4037-9332-9d6285b795b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.737411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add976e4-548f-4037-9332-9d6285b795b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.744611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add976e4-548f-4037-9332-9d6285b795b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.746120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add976e4-548f-4037-9332-9d6285b795b8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.746958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3161f-389b-4012-9a63-98a989802498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3161f-389b-4012-9a63-98a989802498 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.747858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3161f-389b-4012-9a63-98a989802498 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.748121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3161f-389b-4012-9a63-98a989802498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.750657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-add976e4-548f-4037-9332-9d6285b795b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:10.757991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:10.758945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 255/509] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:10.764817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add976e4-548f-4037-9332-9d6285b795b8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.766083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3161f-389b-4012-9a63-98a989802498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.766083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3161f-389b-4012-9a63-98a989802498 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-35b3161f-389b-4012-9a63-98a989802498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:10.779524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:10.780547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:10.781447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:10.781447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:10.781447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:10.781447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 255/510] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:10.782313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3161f-389b-4012-9a63-98a989802498 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.862243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add976e4-548f-4037-9332-9d6285b795b8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.879720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3161f-389b-4012-9a63-98a989802498 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:10.895782 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce1d07-43d9-48fc-9011-e9b5a9fbbea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.896390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce1d07-43d9-48fc-9011-e9b5a9fbbea5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.896671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce1d07-43d9-48fc-9011-e9b5a9fbbea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.896977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce1d07-43d9-48fc-9011-e9b5a9fbbea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.912419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc24ef-bdb6-4f8c-8a92-c8ed48aba0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:10.913049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc24ef-bdb6-4f8c-8a92-c8ed48aba0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:10.913332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc24ef-bdb6-4f8c-8a92-c8ed48aba0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:10.913601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc24ef-bdb6-4f8c-8a92-c8ed48aba0a2 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:10.914037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:10.914316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc24ef-bdb6-4f8c-8a92-c8ed48aba0a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:10.914621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc24ef-bdb6-4f8c-8a92-c8ed48aba0a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.916053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc24ef-bdb6-4f8c-8a92-c8ed48aba0a2 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:10.953552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 256/511] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:10.954406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc24ef-bdb6-4f8c-8a92-c8ed48aba0a2 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:10.965759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce1d07-43d9-48fc-9011-e9b5a9fbbea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:10.967245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce1d07-43d9-48fc-9011-e9b5a9fbbea5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.001798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:11.056246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc24ef-bdb6-4f8c-8a92-c8ed48aba0a2 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.067218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 256/512] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:11.068067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce1d07-43d9-48fc-9011-e9b5a9fbbea5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.090490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fbd55-4dd4-4497-a954-e882cb9e3aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:11.091098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fbd55-4dd4-4497-a954-e882cb9e3aee None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:11.091381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fbd55-4dd4-4497-a954-e882cb9e3aee None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:11.091655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fbd55-4dd4-4497-a954-e882cb9e3aee None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:11.092297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fbd55-4dd4-4497-a954-e882cb9e3aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:11.092604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fbd55-4dd4-4497-a954-e882cb9e3aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:11.094185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fbd55-4dd4-4497-a954-e882cb9e3aee None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.105128 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 257/513] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:11.105910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fbd55-4dd4-4497-a954-e882cb9e3aee None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.119366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:11.169068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce1d07-43d9-48fc-9011-e9b5a9fbbea5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.204268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee654d-b23d-42ab-807c-d4d2011d059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:11.204899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee654d-b23d-42ab-807c-d4d2011d059b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:11.205184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee654d-b23d-42ab-807c-d4d2011d059b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:11.205462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee654d-b23d-42ab-807c-d4d2011d059b None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:11.206180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee654d-b23d-42ab-807c-d4d2011d059b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:11.206487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee654d-b23d-42ab-807c-d4d2011d059b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:11.207952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee654d-b23d-42ab-807c-d4d2011d059b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.208068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fbd55-4dd4-4497-a954-e882cb9e3aee None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.210424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:11.226678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 257/514] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Sep 15 10:43:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:11.227491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee654d-b23d-42ab-807c-d4d2011d059b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.243441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846b332-2731-41e6-a688-fd3975a28deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:11.244048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846b332-2731-41e6-a688-fd3975a28deb None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:11.244329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846b332-2731-41e6-a688-fd3975a28deb None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:11.244601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846b332-2731-41e6-a688-fd3975a28deb None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:11.245269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846b332-2731-41e6-a688-fd3975a28deb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:11.245572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846b332-2731-41e6-a688-fd3975a28deb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:11.246953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846b332-2731-41e6-a688-fd3975a28deb None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.259240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 258/515] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:11.259964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846b332-2731-41e6-a688-fd3975a28deb None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.330337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee654d-b23d-42ab-807c-d4d2011d059b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.337436 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:11.363988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7f8bc-fab9-43f7-a989-5af7deac1c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:11.364600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7f8bc-fab9-43f7-a989-5af7deac1c97 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:11.364879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7f8bc-fab9-43f7-a989-5af7deac1c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:11.365149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7f8bc-fab9-43f7-a989-5af7deac1c97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:11.369235 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846b332-2731-41e6-a688-fd3975a28deb None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.406545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8d140-4b67-42e1-82ec-9e1bd20541f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:11.407243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8d140-4b67-42e1-82ec-9e1bd20541f4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:11.407520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8d140-4b67-42e1-82ec-9e1bd20541f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:11.407785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8d140-4b67-42e1-82ec-9e1bd20541f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:11.418185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8d140-4b67-42e1-82ec-9e1bd20541f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:11.419675 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8d140-4b67-42e1-82ec-9e1bd20541f4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.434067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7f8bc-fab9-43f7-a989-5af7deac1c97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-c8c8d140-4b67-42e1-82ec-9e1bd20541f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:11.434531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:11.435467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:11.436328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:11.436328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:11.436328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:11.436504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7f8bc-fab9-43f7-a989-5af7deac1c97 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.436573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 259/516] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:11.436573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8d140-4b67-42e1-82ec-9e1bd20541f4 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.457075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:11.535071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 258/517] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:11.535845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7f8bc-fab9-43f7-a989-5af7deac1c97 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.539206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8d140-4b67-42e1-82ec-9e1bd20541f4 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.544209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:11.574506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87cec4-4ef9-4fbf-ab72-a31cf3c38a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:11.575108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87cec4-4ef9-4fbf-ab72-a31cf3c38a38 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:11.575393 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87cec4-4ef9-4fbf-ab72-a31cf3c38a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:11.575660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87cec4-4ef9-4fbf-ab72-a31cf3c38a38 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:11.576298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87cec4-4ef9-4fbf-ab72-a31cf3c38a38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:11.578075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87cec4-4ef9-4fbf-ab72-a31cf3c38a38 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.598841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 260/518] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:43:11 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:11.599655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87cec4-4ef9-4fbf-ab72-a31cf3c38a38 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.648013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7f8bc-fab9-43f7-a989-5af7deac1c97 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.688292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c16249-6536-4eb4-9c76-e93170a65e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:11.688950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c16249-6536-4eb4-9c76-e93170a65e2c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:11.689230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c16249-6536-4eb4-9c76-e93170a65e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:11.690297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c16249-6536-4eb4-9c76-e93170a65e2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:11.690894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:11.726835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87cec4-4ef9-4fbf-ab72-a31cf3c38a38 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.766420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0c97d-4fba-48d0-9e2b-547d94fe39df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:11.767446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0c97d-4fba-48d0-9e2b-547d94fe39df None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:11.767958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0c97d-4fba-48d0-9e2b-547d94fe39df None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:11.768229 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0c97d-4fba-48d0-9e2b-547d94fe39df None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:11.769932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c16249-6536-4eb4-9c76-e93170a65e2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:11.771423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c16249-6536-4eb4-9c76-e93170a65e2c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.777865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0c97d-4fba-48d0-9e2b-547d94fe39df None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:11.779210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0c97d-4fba-48d0-9e2b-547d94fe39df None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.780435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-ddf0c97d-4fba-48d0-9e2b-547d94fe39df None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:11.794366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:11.795281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:11.796219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 261/519] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:11.798036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0c97d-4fba-48d0-9e2b-547d94fe39df None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.885597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 259/520] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:11.886400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c16249-6536-4eb4-9c76-e93170a65e2c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.926954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0c97d-4fba-48d0-9e2b-547d94fe39df None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.966424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:11.974021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d61def-7633-4b34-aa69-18503ca4c86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:11.974643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d61def-7633-4b34-aa69-18503ca4c86d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:11.974926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d61def-7633-4b34-aa69-18503ca4c86d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:11.975198 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d61def-7633-4b34-aa69-18503ca4c86d None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:11.975836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d61def-7633-4b34-aa69-18503ca4c86d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:11.976627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d61def-7633-4b34-aa69-18503ca4c86d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:11.978074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d61def-7633-4b34-aa69-18503ca4c86d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:11.988662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c16249-6536-4eb4-9c76-e93170a65e2c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:11.995584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 262/521] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:11.996850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d61def-7633-4b34-aa69-18503ca4c86d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:11.997300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:12.027315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e2388-ac6e-4a3b-ad73-9f859fac01b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.027897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e2388-ac6e-4a3b-ad73-9f859fac01b0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.028176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e2388-ac6e-4a3b-ad73-9f859fac01b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.028447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e2388-ac6e-4a3b-ad73-9f859fac01b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.039620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e2388-ac6e-4a3b-ad73-9f859fac01b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.041108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e2388-ac6e-4a3b-ad73-9f859fac01b0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-4f1e2388-ac6e-4a3b-ad73-9f859fac01b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:12.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:12.057960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:12.058835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:12.058835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:12.058835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:12.058835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 260/522] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:12.058835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e2388-ac6e-4a3b-ad73-9f859fac01b0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.112510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d61def-7633-4b34-aa69-18503ca4c86d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:12.150203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99e65c-a4e3-4ce3-9edc-c8050577f45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.150804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99e65c-a4e3-4ce3-9edc-c8050577f45d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.151083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99e65c-a4e3-4ce3-9edc-c8050577f45d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.151351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99e65c-a4e3-4ce3-9edc-c8050577f45d None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.152071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99e65c-a4e3-4ce3-9edc-c8050577f45d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:12.152374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99e65c-a4e3-4ce3-9edc-c8050577f45d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.153834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99e65c-a4e3-4ce3-9edc-c8050577f45d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.163439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e2388-ac6e-4a3b-ad73-9f859fac01b0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:12.176238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 263/523] 10.222.0.50 () {56 vars in 1139 bytes} [Thu Sep 15 10:43:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:12.176238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99e65c-a4e3-4ce3-9edc-c8050577f45d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.207060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76254d9-8cc3-450f-a74d-697e707a817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.207695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76254d9-8cc3-450f-a74d-697e707a817c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.207978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76254d9-8cc3-450f-a74d-697e707a817c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.208252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76254d9-8cc3-450f-a74d-697e707a817c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.218915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76254d9-8cc3-450f-a74d-697e707a817c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.220386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76254d9-8cc3-450f-a74d-697e707a817c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.229385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b76254d9-8cc3-450f-a74d-697e707a817c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:12.234904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:12.235846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:12.236715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:12.236715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:12.236715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:12.240565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 261/524] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:12.241404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76254d9-8cc3-450f-a74d-697e707a817c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.255116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:12.286326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99e65c-a4e3-4ce3-9edc-c8050577f45d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:12.324960 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05936e23-ce85-4cc4-a204-2afdef1f1878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.325519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05936e23-ce85-4cc4-a204-2afdef1f1878 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.325803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05936e23-ce85-4cc4-a204-2afdef1f1878 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.326442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05936e23-ce85-4cc4-a204-2afdef1f1878 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.334327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05936e23-ce85-4cc4-a204-2afdef1f1878 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.336106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05936e23-ce85-4cc4-a204-2afdef1f1878 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-05936e23-ce85-4cc4-a204-2afdef1f1878 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:12.348629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:12.349695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:12.355731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:12.355731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:12.355731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:12.355731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:12.355731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:12.355731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:12.355731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:12.355731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:12.356176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76254d9-8cc3-450f-a74d-697e707a817c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:12.356256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 264/525] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:12.356256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05936e23-ce85-4cc4-a204-2afdef1f1878 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.393519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf0dc3-31c4-4893-a587-108498507729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.394173 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf0dc3-31c4-4893-a587-108498507729 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.394419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf0dc3-31c4-4893-a587-108498507729 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.394695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf0dc3-31c4-4893-a587-108498507729 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.395371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf0dc3-31c4-4893-a587-108498507729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:12.395703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf0dc3-31c4-4893-a587-108498507729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.397201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf0dc3-31c4-4893-a587-108498507729 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.410290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 262/526] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:12.411120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf0dc3-31c4-4893-a587-108498507729 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.468525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05936e23-ce85-4cc4-a204-2afdef1f1878 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:12.479709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:12.490056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:12.507718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61ad70-025d-4164-845e-324f9d2e37ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.508331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61ad70-025d-4164-845e-324f9d2e37ee None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.508606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61ad70-025d-4164-845e-324f9d2e37ee None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.509007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61ad70-025d-4164-845e-324f9d2e37ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.526563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf0dc3-31c4-4893-a587-108498507729 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:12.570853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685dbe3-0dd4-4c2c-a47d-5e0bc6ddfec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.571579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685dbe3-0dd4-4c2c-a47d-5e0bc6ddfec8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.571865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685dbe3-0dd4-4c2c-a47d-5e0bc6ddfec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.572124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685dbe3-0dd4-4c2c-a47d-5e0bc6ddfec8 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.572766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685dbe3-0dd4-4c2c-a47d-5e0bc6ddfec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:12.573100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685dbe3-0dd4-4c2c-a47d-5e0bc6ddfec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.574571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685dbe3-0dd4-4c2c-a47d-5e0bc6ddfec8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.586695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 263/527] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:12.587423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685dbe3-0dd4-4c2c-a47d-5e0bc6ddfec8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.592202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61ad70-025d-4164-845e-324f9d2e37ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.594602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61ad70-025d-4164-845e-324f9d2e37ee None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.691209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685dbe3-0dd4-4c2c-a47d-5e0bc6ddfec8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:12.702682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 265/528] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:12.702682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61ad70-025d-4164-845e-324f9d2e37ee None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.712164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:12.726981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:12.733527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846d5dd-3718-456c-ad83-0c4bf5848136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.734856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846d5dd-3718-456c-ad83-0c4bf5848136 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.735088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846d5dd-3718-456c-ad83-0c4bf5848136 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.735364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846d5dd-3718-456c-ad83-0c4bf5848136 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.736053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846d5dd-3718-456c-ad83-0c4bf5848136 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:12.736359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846d5dd-3718-456c-ad83-0c4bf5848136 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.737826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846d5dd-3718-456c-ad83-0c4bf5848136 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.759854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 264/529] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:12.759854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846d5dd-3718-456c-ad83-0c4bf5848136 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.815117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61ad70-025d-4164-845e-324f9d2e37ee None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:12.852371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896a2fb-da28-466c-8765-98fe52d7d656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.852952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896a2fb-da28-466c-8765-98fe52d7d656 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.853228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896a2fb-da28-466c-8765-98fe52d7d656 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.853495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896a2fb-da28-466c-8765-98fe52d7d656 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.860195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896a2fb-da28-466c-8765-98fe52d7d656 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.861687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896a2fb-da28-466c-8765-98fe52d7d656 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.864302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846d5dd-3718-456c-ad83-0c4bf5848136 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-4896a2fb-da28-466c-8765-98fe52d7d656 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:12.872758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:12.873680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:12.874559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:12.874559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:12.874559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:12.874559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:12.874559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:12.874559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:12.874559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:12.874559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:12.874559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 266/530] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:12.875857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896a2fb-da28-466c-8765-98fe52d7d656 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.904606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbd669-d106-4a1a-a43c-8d77b05bb641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:12.905377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbd669-d106-4a1a-a43c-8d77b05bb641 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:12.905669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbd669-d106-4a1a-a43c-8d77b05bb641 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:12.906024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbd669-d106-4a1a-a43c-8d77b05bb641 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:12.913204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbd669-d106-4a1a-a43c-8d77b05bb641 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:12.914690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbd669-d106-4a1a-a43c-8d77b05bb641 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6edbd669-d106-4a1a-a43c-8d77b05bb641 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:12.926944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:12.927823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:12.928691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:12.928691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:12.928691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:12.928691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:12.928691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:12.928691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:12.928691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:12.928691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:12.928691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 265/531] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:12.929489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbd669-d106-4a1a-a43c-8d77b05bb641 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:12.958293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:12.986853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:12.995591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896a2fb-da28-466c-8765-98fe52d7d656 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:13.031956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d2c90-b885-49dd-ac60-c260c3efb69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:13.032589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d2c90-b885-49dd-ac60-c260c3efb69a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.032903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d2c90-b885-49dd-ac60-c260c3efb69a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.033186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d2c90-b885-49dd-ac60-c260c3efb69a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.044003 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d2c90-b885-49dd-ac60-c260c3efb69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:13.045509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d2c90-b885-49dd-ac60-c260c3efb69a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:13.049343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbd669-d106-4a1a-a43c-8d77b05bb641 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-087d2c90-b885-49dd-ac60-c260c3efb69a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:13.061755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:13.063272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:13.069201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:13.069201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:13.069201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:13.069201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 267/532] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:13.069201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d2c90-b885-49dd-ac60-c260c3efb69a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:13.087012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65628631-7999-4411-a119-82438f6f06c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:13.087624 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65628631-7999-4411-a119-82438f6f06c2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.087907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65628631-7999-4411-a119-82438f6f06c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.088185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65628631-7999-4411-a119-82438f6f06c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.088937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65628631-7999-4411-a119-82438f6f06c2 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=218a015dd2b34435a76ecc8a763562ed) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:13.089252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65628631-7999-4411-a119-82438f6f06c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:13.090721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65628631-7999-4411-a119-82438f6f06c2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:13.096387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 266/533] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:12 2022] GET /identity/v3/role_assignments?user.id=218a015dd2b34435a76ecc8a763562ed&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:13.101820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65628631-7999-4411-a119-82438f6f06c2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:13.197313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d2c90-b885-49dd-ac60-c260c3efb69a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:13.233297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:13.252280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391a1e8-920f-4c22-aa36-3ce2deb1a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:13.253097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391a1e8-920f-4c22-aa36-3ce2deb1a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.253471 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391a1e8-920f-4c22-aa36-3ce2deb1a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.253840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391a1e8-920f-4c22-aa36-3ce2deb1a9e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.254709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391a1e8-920f-4c22-aa36-3ce2deb1a9e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:13.255117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391a1e8-920f-4c22-aa36-3ce2deb1a9e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:13.255454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65628631-7999-4411-a119-82438f6f06c2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:13.256972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391a1e8-920f-4c22-aa36-3ce2deb1a9e0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:13.273997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:13.288697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 268/534] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:13.288697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391a1e8-920f-4c22-aa36-3ce2deb1a9e0 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:13.302541 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f39b74-3048-4d28-9fc8-d2ab768cc9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:13.303299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f39b74-3048-4d28-9fc8-d2ab768cc9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.303686 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f39b74-3048-4d28-9fc8-d2ab768cc9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.304082 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f39b74-3048-4d28-9fc8-d2ab768cc9e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.305044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f39b74-3048-4d28-9fc8-d2ab768cc9e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:13.305511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f39b74-3048-4d28-9fc8-d2ab768cc9e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:13.307113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f39b74-3048-4d28-9fc8-d2ab768cc9e4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:13.324605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 267/535] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:13.325683 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f39b74-3048-4d28-9fc8-d2ab768cc9e4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:13.407618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391a1e8-920f-4c22-aa36-3ce2deb1a9e0 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:13.438967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f39b74-3048-4d28-9fc8-d2ab768cc9e4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:13.444830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9888d1-478d-4ec0-9d04-081a88415c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:13.445463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9888d1-478d-4ec0-9d04-081a88415c52 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.445758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9888d1-478d-4ec0-9d04-081a88415c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.446060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9888d1-478d-4ec0-9d04-081a88415c52 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.446712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9888d1-478d-4ec0-9d04-081a88415c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:13.447015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9888d1-478d-4ec0-9d04-081a88415c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:13.448575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9888d1-478d-4ec0-9d04-081a88415c52 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:13.462741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 269/536] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:13.463850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9888d1-478d-4ec0-9d04-081a88415c52 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:13.487300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79f711-e2ec-44dd-bb45-4168b9064d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:13.488054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79f711-e2ec-44dd-bb45-4168b9064d95 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.488407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79f711-e2ec-44dd-bb45-4168b9064d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.489256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79f711-e2ec-44dd-bb45-4168b9064d95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.490076 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79f711-e2ec-44dd-bb45-4168b9064d95 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=6908a326a3d3437c9db898976a2def1b) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:13.490523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79f711-e2ec-44dd-bb45-4168b9064d95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:13.492685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79f711-e2ec-44dd-bb45-4168b9064d95 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:13.501738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 268/537] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:13 2022] GET /identity/v3/role_assignments?user.id=6908a326a3d3437c9db898976a2def1b&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:13.502855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79f711-e2ec-44dd-bb45-4168b9064d95 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:13.519796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:13.568618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:13.627030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9888d1-478d-4ec0-9d04-081a88415c52 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:13.678565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90629a3a-4c2d-43ef-a186-71cc6bc03605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:13.679485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90629a3a-4c2d-43ef-a186-71cc6bc03605 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.679939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90629a3a-4c2d-43ef-a186-71cc6bc03605 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.680597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90629a3a-4c2d-43ef-a186-71cc6bc03605 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.682307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90629a3a-4c2d-43ef-a186-71cc6bc03605 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=d7fa5d550a2d442a820cf4e12ebc36c5) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:13.683047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90629a3a-4c2d-43ef-a186-71cc6bc03605 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:13.685550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90629a3a-4c2d-43ef-a186-71cc6bc03605 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:13.700385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 270/538] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:13 2022] GET /identity/v3/role_assignments?user.id=d7fa5d550a2d442a820cf4e12ebc36c5&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:13.727389 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79f711-e2ec-44dd-bb45-4168b9064d95 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:13.776769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a4999-80e1-4385-a384-4892e63464b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:13.777761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a4999-80e1-4385-a384-4892e63464b0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.778218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a4999-80e1-4385-a384-4892e63464b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.778663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a4999-80e1-4385-a384-4892e63464b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.787368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a4999-80e1-4385-a384-4892e63464b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:13.789164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a4999-80e1-4385-a384-4892e63464b0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-fa6a4999-80e1-4385-a384-4892e63464b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:13.802838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:13.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:13.804639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:13.804639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:13.804639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:13.804639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:13.804639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:13.804639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:13.804639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:13.804639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:13.806892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 269/539] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:13.813755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90629a3a-4c2d-43ef-a186-71cc6bc03605 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:13.854818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62c64-a0f4-4f40-b6e7-4f835483fb83 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.855144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62c64-a0f4-4f40-b6e7-4f835483fb83 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.855371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62c64-a0f4-4f40-b6e7-4f835483fb83 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.856898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 270/540] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:13.863342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeacf2-54e5-41d1-a2b4-0c7706269776 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.863967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeacf2-54e5-41d1-a2b4-0c7706269776 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.864449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeacf2-54e5-41d1-a2b4-0c7706269776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.884507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d0eeacf2-54e5-41d1-a2b4-0c7706269776 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:13.886091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d0eeacf2-54e5-41d1-a2b4-0c7706269776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:13.940816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90629a3a-4c2d-43ef-a186-71cc6bc03605 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:13.953309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:13.977261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 271/541] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:43:13.994416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89de2c-18de-4e96-af6d-73ad3525f415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:13.995036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89de2c-18de-4e96-af6d-73ad3525f415 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:13.995311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89de2c-18de-4e96-af6d-73ad3525f415 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:13.995575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89de2c-18de-4e96-af6d-73ad3525f415 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:13.996266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89de2c-18de-4e96-af6d-73ad3525f415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:13.996566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89de2c-18de-4e96-af6d-73ad3525f415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:13.998570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89de2c-18de-4e96-af6d-73ad3525f415 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:14.017189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 271/542] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:14.023055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eeacf2-54e5-41d1-a2b4-0c7706269776 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:14.053594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:14.158058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eeacf2-54e5-41d1-a2b4-0c7706269776 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:14.210150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a095f-eb18-4ef6-8939-7f97ee168899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:14.210691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a095f-eb18-4ef6-8939-7f97ee168899 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:14.211029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a095f-eb18-4ef6-8939-7f97ee168899 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:14.211307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a095f-eb18-4ef6-8939-7f97ee168899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:14.212031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a095f-eb18-4ef6-8939-7f97ee168899 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=f5e7b08dc3544e76ac6732ca2aa89bf5) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:14.212337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a095f-eb18-4ef6-8939-7f97ee168899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:14.213801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a095f-eb18-4ef6-8939-7f97ee168899 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:14.225120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 272/543] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:14 2022] GET /identity/v3/role_assignments?user.id=f5e7b08dc3544e76ac6732ca2aa89bf5&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:14.361394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89de2c-18de-4e96-af6d-73ad3525f415 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:14.500512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89de2c-18de-4e96-af6d-73ad3525f415 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:14.539384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670cd73-17d9-44f8-bbcb-090d5d02fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:14.539384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670cd73-17d9-44f8-bbcb-090d5d02fc11 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:14.539384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670cd73-17d9-44f8-bbcb-090d5d02fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:14.539384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670cd73-17d9-44f8-bbcb-090d5d02fc11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:14.549467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670cd73-17d9-44f8-bbcb-090d5d02fc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-b670cd73-17d9-44f8-bbcb-090d5d02fc11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:14.554639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:14.555536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:14.556350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 272/544] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:14.564823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a095f-eb18-4ef6-8939-7f97ee168899 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:14.672843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:14.683783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a095f-eb18-4ef6-8939-7f97ee168899 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:14.722127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e6114-0cd6-4f19-9f6c-3147b386b60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:14.722747 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e6114-0cd6-4f19-9f6c-3147b386b60a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:14.723035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e6114-0cd6-4f19-9f6c-3147b386b60a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:14.723308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e6114-0cd6-4f19-9f6c-3147b386b60a None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:14.723976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e6114-0cd6-4f19-9f6c-3147b386b60a None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:14.724279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e6114-0cd6-4f19-9f6c-3147b386b60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:14.725759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e6114-0cd6-4f19-9f6c-3147b386b60a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:14.734244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 273/545] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:14.740165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670cd73-17d9-44f8-bbcb-090d5d02fc11 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:14.775080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:14.867056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670cd73-17d9-44f8-bbcb-090d5d02fc11 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:14.902461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1150ca-b505-4755-a88b-3aa234b64bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:14.903089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1150ca-b505-4755-a88b-3aa234b64bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:14.903377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1150ca-b505-4755-a88b-3aa234b64bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:14.903705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1150ca-b505-4755-a88b-3aa234b64bf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:14.915194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1150ca-b505-4755-a88b-3aa234b64bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:14.916666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1150ca-b505-4755-a88b-3aa234b64bf0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-3b1150ca-b505-4755-a88b-3aa234b64bf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:14.932655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:14.933618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:14.934474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:14.934474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:14.934474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:14.934474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 273/546] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:14.939912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e6114-0cd6-4f19-9f6c-3147b386b60a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:15.071684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e6114-0cd6-4f19-9f6c-3147b386b60a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:15.115175 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdce2c-7813-4099-a9bd-6e80d50f720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:15.115800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdce2c-7813-4099-a9bd-6e80d50f720e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:15.116081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdce2c-7813-4099-a9bd-6e80d50f720e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:15.116355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdce2c-7813-4099-a9bd-6e80d50f720e None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:15.117027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdce2c-7813-4099-a9bd-6e80d50f720e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:15.117333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdce2c-7813-4099-a9bd-6e80d50f720e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:15.118721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdce2c-7813-4099-a9bd-6e80d50f720e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:15.131901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 274/547] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:15.137215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1150ca-b505-4755-a88b-3aa234b64bf0 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:15.252961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:15.276844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1150ca-b505-4755-a88b-3aa234b64bf0 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:15.313463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3665091-a8fa-4594-b0f2-24836b060cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:15.314096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3665091-a8fa-4594-b0f2-24836b060cff None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:15.314374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3665091-a8fa-4594-b0f2-24836b060cff None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:15.314648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3665091-a8fa-4594-b0f2-24836b060cff None admin] PATH_INFO: `/v3/users/swift` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:15.322638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3665091-a8fa-4594-b0f2-24836b060cff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:15.324203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3665091-a8fa-4594-b0f2-24836b060cff None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-a3665091-a8fa-4594-b0f2-24836b060cff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:15.336039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:15.337476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:15.338710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 274/548] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:15.343296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdce2c-7813-4099-a9bd-6e80d50f720e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:15.378119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:15.504412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdce2c-7813-4099-a9bd-6e80d50f720e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:15.507390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb87a1-3b04-4085-a471-07b208f55367 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:15.507916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb87a1-3b04-4085-a471-07b208f55367 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:15.508411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb87a1-3b04-4085-a471-07b208f55367 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:15.510265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 275/549] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:43:15.516042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1d009-7aa5-4a88-8605-3e07c8b74f79 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:15.517618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1d009-7aa5-4a88-8605-3e07c8b74f79 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:15.518103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1d009-7aa5-4a88-8605-3e07c8b74f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:15.556116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-eec1d009-7aa5-4a88-8605-3e07c8b74f79 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:15.557373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-eec1d009-7aa5-4a88-8605-3e07c8b74f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:15.583194 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd89c5a-4aea-4e7c-9a44-570438bb2a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:15.583846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd89c5a-4aea-4e7c-9a44-570438bb2a3b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:15.584025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd89c5a-4aea-4e7c-9a44-570438bb2a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:15.584286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd89c5a-4aea-4e7c-9a44-570438bb2a3b None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:15.585067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd89c5a-4aea-4e7c-9a44-570438bb2a3b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:15.585379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd89c5a-4aea-4e7c-9a44-570438bb2a3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:15.586862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd89c5a-4aea-4e7c-9a44-570438bb2a3b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:15.616119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 275/550] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Sep 15 10:43:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:15.617380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd89c5a-4aea-4e7c-9a44-570438bb2a3b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:15.686032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 276/551] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:43:15.725348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:15.756456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd89c5a-4aea-4e7c-9a44-570438bb2a3b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:15.792580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb6acd-d6ac-4e20-9892-6e32b4735960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:15.793373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb6acd-d6ac-4e20-9892-6e32b4735960 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:15.793658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb6acd-d6ac-4e20-9892-6e32b4735960 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:15.793937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb6acd-d6ac-4e20-9892-6e32b4735960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:15.865669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb6acd-d6ac-4e20-9892-6e32b4735960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:15.867085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb6acd-d6ac-4e20-9892-6e32b4735960 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:15.957990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:15.976658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 276/552] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:15.978055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1d009-7aa5-4a88-8605-3e07c8b74f79 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:16.057818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:16.094563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1d009-7aa5-4a88-8605-3e07c8b74f79 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:16.111875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb6acd-d6ac-4e20-9892-6e32b4735960 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:16.138895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab4b79-ba29-4407-9f40-8e159dfff606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:16.138895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab4b79-ba29-4407-9f40-8e159dfff606 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.138895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab4b79-ba29-4407-9f40-8e159dfff606 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.138895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab4b79-ba29-4407-9f40-8e159dfff606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.148326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab4b79-ba29-4407-9f40-8e159dfff606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:16.150016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab4b79-ba29-4407-9f40-8e159dfff606 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-e6ab4b79-ba29-4407-9f40-8e159dfff606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:16.174186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:16.175254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:16.176120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:16.176120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:16.176120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:16.185912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 277/553] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:16.193841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab4b79-ba29-4407-9f40-8e159dfff606 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:16.304248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb6acd-d6ac-4e20-9892-6e32b4735960 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:16.345818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15103449-e9cc-4dd4-bc9b-84e832eb22bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:16.346534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15103449-e9cc-4dd4-bc9b-84e832eb22bf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.346819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15103449-e9cc-4dd4-bc9b-84e832eb22bf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.347091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15103449-e9cc-4dd4-bc9b-84e832eb22bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.368378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15103449-e9cc-4dd4-bc9b-84e832eb22bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-15103449-e9cc-4dd4-bc9b-84e832eb22bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:16.376627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:16.377572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:16.378474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:16.380559 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 277/554] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:16.385930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15103449-e9cc-4dd4-bc9b-84e832eb22bf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:16.394928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab4b79-ba29-4407-9f40-8e159dfff606 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:16.436382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:16.455739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:16.462399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e14231-ff99-4e37-b080-fa78f33654ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:16.463325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e14231-ff99-4e37-b080-fa78f33654ea None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.463753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e14231-ff99-4e37-b080-fa78f33654ea None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.464168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e14231-ff99-4e37-b080-fa78f33654ea None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.465006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e14231-ff99-4e37-b080-fa78f33654ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:16.465458 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e14231-ff99-4e37-b080-fa78f33654ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:16.467760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e14231-ff99-4e37-b080-fa78f33654ea None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:16.485214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 278/555] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:16.487311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e28ba-e77c-42e7-ac49-419be3dce7af None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.487746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e28ba-e77c-42e7-ac49-419be3dce7af None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.488411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e28ba-e77c-42e7-ac49-419be3dce7af None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.494299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 279/556] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:16.494299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e28ba-e77c-42e7-ac49-419be3dce7af None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:16.564688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15103449-e9cc-4dd4-bc9b-84e832eb22bf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:16.603156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ae789-9789-410f-bfd4-63f015a8be4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:16.603839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ae789-9789-410f-bfd4-63f015a8be4d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.604041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ae789-9789-410f-bfd4-63f015a8be4d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.604311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ae789-9789-410f-bfd4-63f015a8be4d None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.604998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ae789-9789-410f-bfd4-63f015a8be4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:16.605299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ae789-9789-410f-bfd4-63f015a8be4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:16.606675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ae789-9789-410f-bfd4-63f015a8be4d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:16.610121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e28ba-e77c-42e7-ac49-419be3dce7af None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:16.619306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 278/557] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:16.619306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2b69-8c04-42d0-aaed-570f3b76a8c7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.619306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2b69-8c04-42d0-aaed-570f3b76a8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.619306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2b69-8c04-42d0-aaed-570f3b76a8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.635891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bfcf2b69-8c04-42d0-aaed-570f3b76a8c7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:16.637120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bfcf2b69-8c04-42d0-aaed-570f3b76a8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:16.654184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818c592-5025-4cf7-92c4-559ff4d241ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:16.654822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818c592-5025-4cf7-92c4-559ff4d241ef None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.655339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818c592-5025-4cf7-92c4-559ff4d241ef None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.655339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818c592-5025-4cf7-92c4-559ff4d241ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.662885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818c592-5025-4cf7-92c4-559ff4d241ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:16.664331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818c592-5025-4cf7-92c4-559ff4d241ef None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-e818c592-5025-4cf7-92c4-559ff4d241ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:16.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:16.678246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:16.679171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:16.679171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:16.679171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:16.679171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:16.679171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:16.679171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:16.679171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:16.679171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:16.683124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 280/558] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:16.684246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93382215-d2b3-4e9c-b7e2-5f16c26d66f2 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.684516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93382215-d2b3-4e9c-b7e2-5f16c26d66f2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.684793 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93382215-d2b3-4e9c-b7e2-5f16c26d66f2 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.686084 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 281/559] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:16.686738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93382215-d2b3-4e9c-b7e2-5f16c26d66f2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:16.717435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 279/560] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:16.726007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf2b69-8c04-42d0-aaed-570f3b76a8c7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:16.736732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:16.759363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:16.800357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93382215-d2b3-4e9c-b7e2-5f16c26d66f2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:16.837926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258b99-cf67-4cdb-9a66-6b28db0957d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:16.838979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258b99-cf67-4cdb-9a66-6b28db0957d6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.839469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258b99-cf67-4cdb-9a66-6b28db0957d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.839963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258b99-cf67-4cdb-9a66-6b28db0957d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.852171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258b99-cf67-4cdb-9a66-6b28db0957d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:16.854022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258b99-cf67-4cdb-9a66-6b28db0957d6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:16.857689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf2b69-8c04-42d0-aaed-570f3b76a8c7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-52258b99-cf67-4cdb-9a66-6b28db0957d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:16.869624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:16.870639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:16.871581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:16.871581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:16.871581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:16.878199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 282/561] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:16.879799 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc567d9-554e-4d27-aae8-e0ce96ba324f None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.880295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc567d9-554e-4d27-aae8-e0ce96ba324f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.880777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc567d9-554e-4d27-aae8-e0ce96ba324f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.897137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee77875-a078-4e42-8501-0ca5834876f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:16.898004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee77875-a078-4e42-8501-0ca5834876f9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:16.898284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee77875-a078-4e42-8501-0ca5834876f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:16.898562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee77875-a078-4e42-8501-0ca5834876f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:16.899257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee77875-a078-4e42-8501-0ca5834876f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:16.899564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee77875-a078-4e42-8501-0ca5834876f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:16.901032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee77875-a078-4e42-8501-0ca5834876f9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:16.902905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2dc567d9-554e-4d27-aae8-e0ce96ba324f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:16.904477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2dc567d9-554e-4d27-aae8-e0ce96ba324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:16.917642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 280/562] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:16.918912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee77875-a078-4e42-8501-0ca5834876f9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:16.997155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 283/563] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:16.998019 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc567d9-554e-4d27-aae8-e0ce96ba324f None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:17.014024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:17.031628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:17.035524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee77875-a078-4e42-8501-0ca5834876f9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:17.072762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a410f00-8e1e-467b-ad0a-c36024fb5cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:17.073743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a410f00-8e1e-467b-ad0a-c36024fb5cba None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.074453 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a410f00-8e1e-467b-ad0a-c36024fb5cba None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.075045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a410f00-8e1e-467b-ad0a-c36024fb5cba None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.075889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a410f00-8e1e-467b-ad0a-c36024fb5cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:17.076411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a410f00-8e1e-467b-ad0a-c36024fb5cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:17.079077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a410f00-8e1e-467b-ad0a-c36024fb5cba None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:17.097188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 281/564] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:17.098454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a410f00-8e1e-467b-ad0a-c36024fb5cba None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:17.126311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc567d9-554e-4d27-aae8-e0ce96ba324f None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:17.178318 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab16076-0e6c-467d-b503-2f21f5b838d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:17.178848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab16076-0e6c-467d-b503-2f21f5b838d5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.179130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab16076-0e6c-467d-b503-2f21f5b838d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.179550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab16076-0e6c-467d-b503-2f21f5b838d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.180420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab16076-0e6c-467d-b503-2f21f5b838d5 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=06007b3b2260409f863ac3d0ac85f4cc) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:17.180732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab16076-0e6c-467d-b503-2f21f5b838d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:17.183353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab16076-0e6c-467d-b503-2f21f5b838d5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:17.190875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 284/565] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:16 2022] GET /identity/v3/role_assignments?user.id=06007b3b2260409f863ac3d0ac85f4cc&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:17.191897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab16076-0e6c-467d-b503-2f21f5b838d5 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:17.219646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a410f00-8e1e-467b-ad0a-c36024fb5cba None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:17.268966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f82e7a-4c14-4db9-bd5c-3ffa32774b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:17.269626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82e7a-4c14-4db9-bd5c-3ffa32774b79 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.269909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82e7a-4c14-4db9-bd5c-3ffa32774b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.270188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82e7a-4c14-4db9-bd5c-3ffa32774b79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.283839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f82e7a-4c14-4db9-bd5c-3ffa32774b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-19f82e7a-4c14-4db9-bd5c-3ffa32774b79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:17.289340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:17.290343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:17.291199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 282/566] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:17.292456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d65cd-23e4-4467-a3e8-15550119d4b5 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.292620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d65cd-23e4-4467-a3e8-15550119d4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.294099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d65cd-23e4-4467-a3e8-15550119d4b5 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.298723 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 283/567] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:17.299591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d65cd-23e4-4467-a3e8-15550119d4b5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:17.305534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:17.318359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:17.339252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab16076-0e6c-467d-b503-2f21f5b838d5 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:17.376238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af44d7-5765-46ed-be3a-0bfc7a9bff76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:17.376914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af44d7-5765-46ed-be3a-0bfc7a9bff76 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.377199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af44d7-5765-46ed-be3a-0bfc7a9bff76 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.377473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af44d7-5765-46ed-be3a-0bfc7a9bff76 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.385686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af44d7-5765-46ed-be3a-0bfc7a9bff76 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:17.387191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af44d7-5765-46ed-be3a-0bfc7a9bff76 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-a2af44d7-5765-46ed-be3a-0bfc7a9bff76 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:17.400329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:17.406361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 285/568] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Sep 15 10:43:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248382d-a5ec-477a-b717-30e97f3c65d5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.407336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248382d-a5ec-477a-b717-30e97f3c65d5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.408058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248382d-a5ec-477a-b717-30e97f3c65d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.426448 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d65cd-23e4-4467-a3e8-15550119d4b5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:17.426944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3248382d-a5ec-477a-b717-30e97f3c65d5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:17.428390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3248382d-a5ec-477a-b717-30e97f3c65d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:17.462130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e7e7b-3017-445b-9258-b66fceed0d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:17.462736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e7e7b-3017-445b-9258-b66fceed0d2b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.463013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e7e7b-3017-445b-9258-b66fceed0d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.463285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e7e7b-3017-445b-9258-b66fceed0d2b None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.463933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e7e7b-3017-445b-9258-b66fceed0d2b None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:17.464235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e7e7b-3017-445b-9258-b66fceed0d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:17.465653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e7e7b-3017-445b-9258-b66fceed0d2b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:17.474300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 284/569] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:17.479321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e7e7b-3017-445b-9258-b66fceed0d2b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:17.505377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 286/570] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:17.514172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248382d-a5ec-477a-b717-30e97f3c65d5 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:17.559470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:17.597898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e7e7b-3017-445b-9258-b66fceed0d2b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:17.629813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:17.635237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf8b74-8987-4d32-9d97-139f0f656d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:17.636394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf8b74-8987-4d32-9d97-139f0f656d70 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.636877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf8b74-8987-4d32-9d97-139f0f656d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.637308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf8b74-8987-4d32-9d97-139f0f656d70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.648597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf8b74-8987-4d32-9d97-139f0f656d70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6ebf8b74-8987-4d32-9d97-139f0f656d70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:17.654260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:17.655387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:17.656757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:17.659100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 285/571] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:17.660126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf8b74-8987-4d32-9d97-139f0f656d70 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:17.661856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248382d-a5ec-477a-b717-30e97f3c65d5 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:17.700696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9087c-e646-4a00-b6d9-37a36acb3bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:17.701648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9087c-e646-4a00-b6d9-37a36acb3bdb None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.702334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9087c-e646-4a00-b6d9-37a36acb3bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.702991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9087c-e646-4a00-b6d9-37a36acb3bdb None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.704051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9087c-e646-4a00-b6d9-37a36acb3bdb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:17.704826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9087c-e646-4a00-b6d9-37a36acb3bdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:17.706426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9087c-e646-4a00-b6d9-37a36acb3bdb None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:17.725772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 287/572] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Sep 15 10:43:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:17.726814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9087c-e646-4a00-b6d9-37a36acb3bdb None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:17.792778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf8b74-8987-4d32-9d97-139f0f656d70 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:17.825299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:17.839287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1025202-c9cf-4d15-b829-70704bf3e4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:17.839921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1025202-c9cf-4d15-b829-70704bf3e4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.840209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1025202-c9cf-4d15-b829-70704bf3e4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.840488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1025202-c9cf-4d15-b829-70704bf3e4d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.855415 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1025202-c9cf-4d15-b829-70704bf3e4d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:17.857560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1025202-c9cf-4d15-b829-70704bf3e4d0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:17.865951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9087c-e646-4a00-b6d9-37a36acb3bdb None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e1025202-c9cf-4d15-b829-70704bf3e4d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:17.876359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:17.877610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:17.878856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:17.878856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:17.878856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:17.880957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 286/573] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:17.881883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1025202-c9cf-4d15-b829-70704bf3e4d0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:17.913019 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfac86-f82d-4f3f-b9d0-655ecb839d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:17.914075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfac86-f82d-4f3f-b9d0-655ecb839d8a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:17.914459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfac86-f82d-4f3f-b9d0-655ecb839d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:17.914842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfac86-f82d-4f3f-b9d0-655ecb839d8a None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:17.915888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfac86-f82d-4f3f-b9d0-655ecb839d8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:17.916595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfac86-f82d-4f3f-b9d0-655ecb839d8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:17.918548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfac86-f82d-4f3f-b9d0-655ecb839d8a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:17.920929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:17.937416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 288/574] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:17.938486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfac86-f82d-4f3f-b9d0-655ecb839d8a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.019361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1025202-c9cf-4d15-b829-70704bf3e4d0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.058973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb89325-c1ee-47ef-b76e-9262fb3e1887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.059640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb89325-c1ee-47ef-b76e-9262fb3e1887 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.059922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb89325-c1ee-47ef-b76e-9262fb3e1887 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.060201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb89325-c1ee-47ef-b76e-9262fb3e1887 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.065241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfac86-f82d-4f3f-b9d0-655ecb839d8a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.079701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb89325-c1ee-47ef-b76e-9262fb3e1887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.081210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb89325-c1ee-47ef-b76e-9262fb3e1887 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ceb89325-c1ee-47ef-b76e-9262fb3e1887 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:18.096711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:18.097874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:18.098834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:18.098834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:18.098834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:18.098834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 287/575] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:18.099070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb89325-c1ee-47ef-b76e-9262fb3e1887 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.127605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481e4fd-7802-472d-b919-902bc7d3b796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.128908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481e4fd-7802-472d-b919-902bc7d3b796 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.129193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481e4fd-7802-472d-b919-902bc7d3b796 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.129777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481e4fd-7802-472d-b919-902bc7d3b796 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.133455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:18.142774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481e4fd-7802-472d-b919-902bc7d3b796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-5481e4fd-7802-472d-b919-902bc7d3b796 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:18.150152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:18.151130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:18.152044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 289/576] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:18.152778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481e4fd-7802-472d-b919-902bc7d3b796 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.215632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb89325-c1ee-47ef-b76e-9262fb3e1887 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.225839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:18.254057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99266cbf-c4f6-432c-8530-5fa4647291d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.255017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99266cbf-c4f6-432c-8530-5fa4647291d0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.255419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99266cbf-c4f6-432c-8530-5fa4647291d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.255775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99266cbf-c4f6-432c-8530-5fa4647291d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.256509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99266cbf-c4f6-432c-8530-5fa4647291d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:18.256963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99266cbf-c4f6-432c-8530-5fa4647291d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.258436 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99266cbf-c4f6-432c-8530-5fa4647291d0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.275665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 288/577] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:18.275665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99266cbf-c4f6-432c-8530-5fa4647291d0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.280617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481e4fd-7802-472d-b919-902bc7d3b796 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.315672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7ff7-8af3-41e4-975e-9e2940c7fc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.316390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7ff7-8af3-41e4-975e-9e2940c7fc4d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.316734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7ff7-8af3-41e4-975e-9e2940c7fc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.317096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7ff7-8af3-41e4-975e-9e2940c7fc4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.327997 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d7ff7-8af3-41e4-975e-9e2940c7fc4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.329566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d7ff7-8af3-41e4-975e-9e2940c7fc4d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-925d7ff7-8af3-41e4-975e-9e2940c7fc4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:18.343924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:18.345245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:18.346178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:18.346178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:18.346178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:18.346178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 290/578] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:18.346439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7ff7-8af3-41e4-975e-9e2940c7fc4d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.377259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:18.388043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99266cbf-c4f6-432c-8530-5fa4647291d0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.424341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848af4d-38c0-4f5c-964e-1b9964a0fb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.424977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848af4d-38c0-4f5c-964e-1b9964a0fb4e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.425257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848af4d-38c0-4f5c-964e-1b9964a0fb4e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.425527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848af4d-38c0-4f5c-964e-1b9964a0fb4e None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.426157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848af4d-38c0-4f5c-964e-1b9964a0fb4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:18.426455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848af4d-38c0-4f5c-964e-1b9964a0fb4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.427820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848af4d-38c0-4f5c-964e-1b9964a0fb4e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.440862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 289/579] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:18.440862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848af4d-38c0-4f5c-964e-1b9964a0fb4e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.460689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7ff7-8af3-41e4-975e-9e2940c7fc4d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.470754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:18.499032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ca8d5-3245-463e-8588-3bc3f29c132c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.500213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ca8d5-3245-463e-8588-3bc3f29c132c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.500526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ca8d5-3245-463e-8588-3bc3f29c132c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.500846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ca8d5-3245-463e-8588-3bc3f29c132c None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.501544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ca8d5-3245-463e-8588-3bc3f29c132c None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:18.501886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ca8d5-3245-463e-8588-3bc3f29c132c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.503310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ca8d5-3245-463e-8588-3bc3f29c132c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.516743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 291/580] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:18.516743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ca8d5-3245-463e-8588-3bc3f29c132c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.558942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848af4d-38c0-4f5c-964e-1b9964a0fb4e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.595460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf38bd-9ebd-4dec-96de-08f61303ad94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.596088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf38bd-9ebd-4dec-96de-08f61303ad94 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.596368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf38bd-9ebd-4dec-96de-08f61303ad94 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.596638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf38bd-9ebd-4dec-96de-08f61303ad94 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.604778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf38bd-9ebd-4dec-96de-08f61303ad94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.606272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf38bd-9ebd-4dec-96de-08f61303ad94 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-77bf38bd-9ebd-4dec-96de-08f61303ad94 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:18.617638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:18.618646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 290/581] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:18.619710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf38bd-9ebd-4dec-96de-08f61303ad94 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.630958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ca8d5-3245-463e-8588-3bc3f29c132c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.634863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:18.666133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b0022-4f3b-48d0-98f2-5a3e0ddce1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.666741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b0022-4f3b-48d0-98f2-5a3e0ddce1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.667027 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b0022-4f3b-48d0-98f2-5a3e0ddce1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.667299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b0022-4f3b-48d0-98f2-5a3e0ddce1b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.667946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b0022-4f3b-48d0-98f2-5a3e0ddce1b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:18.668249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b0022-4f3b-48d0-98f2-5a3e0ddce1b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.669654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b0022-4f3b-48d0-98f2-5a3e0ddce1b0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.682956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 292/582] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:18.683762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b0022-4f3b-48d0-98f2-5a3e0ddce1b0 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.720365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:18.734580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf38bd-9ebd-4dec-96de-08f61303ad94 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.769923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d132a9-a7de-47a7-affd-e710cc89f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.770645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d132a9-a7de-47a7-affd-e710cc89f0ab None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.770999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d132a9-a7de-47a7-affd-e710cc89f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.771342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d132a9-a7de-47a7-affd-e710cc89f0ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.778070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d132a9-a7de-47a7-affd-e710cc89f0ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.779599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d132a9-a7de-47a7-affd-e710cc89f0ab None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-d0d132a9-a7de-47a7-affd-e710cc89f0ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:18.791214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:18.792379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:18.793535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:18.793535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:18.793535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:18.793535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:18.793535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:18.793535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:18.793535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:18.793535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:18.799233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 291/583] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:18.800130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d132a9-a7de-47a7-affd-e710cc89f0ab None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.801647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b0022-4f3b-48d0-98f2-5a3e0ddce1b0 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.840419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82497595-613d-4ae0-b47c-25cdef35cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.841099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82497595-613d-4ae0-b47c-25cdef35cf34 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.841444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82497595-613d-4ae0-b47c-25cdef35cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.841785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82497595-613d-4ae0-b47c-25cdef35cf34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.852888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82497595-613d-4ae0-b47c-25cdef35cf34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.854474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82497595-613d-4ae0-b47c-25cdef35cf34 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-82497595-613d-4ae0-b47c-25cdef35cf34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:18.869175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:18.870379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:18.871392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:18.871392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:18.871392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:18.871392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 293/584] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:18.871930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82497595-613d-4ae0-b47c-25cdef35cf34 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.881320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:18.917074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d132a9-a7de-47a7-affd-e710cc89f0ab None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:18.953533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07bf17-f615-41f7-b3cb-57a0bfa098b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:18.954157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07bf17-f615-41f7-b3cb-57a0bfa098b1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:18.954440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07bf17-f615-41f7-b3cb-57a0bfa098b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:18.954723 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07bf17-f615-41f7-b3cb-57a0bfa098b1 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:18.955453 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07bf17-f615-41f7-b3cb-57a0bfa098b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:18.955762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07bf17-f615-41f7-b3cb-57a0bfa098b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:18.957226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07bf17-f615-41f7-b3cb-57a0bfa098b1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:18.960233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:18.976396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 292/585] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:18.977845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07bf17-f615-41f7-b3cb-57a0bfa098b1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:18.980451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82497595-613d-4ae0-b47c-25cdef35cf34 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.018222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496e1ac-56ab-46da-8d9e-2098e03d90f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.019957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496e1ac-56ab-46da-8d9e-2098e03d90f2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.020418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496e1ac-56ab-46da-8d9e-2098e03d90f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.020885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496e1ac-56ab-46da-8d9e-2098e03d90f2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.035940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496e1ac-56ab-46da-8d9e-2098e03d90f2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.037696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496e1ac-56ab-46da-8d9e-2098e03d90f2 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-4496e1ac-56ab-46da-8d9e-2098e03d90f2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:19.052057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:19.053284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 10:43:19.054507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:19.060250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 294/586] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:19.060250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496e1ac-56ab-46da-8d9e-2098e03d90f2 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.095198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07bf17-f615-41f7-b3cb-57a0bfa098b1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.122701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:19.131366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d69e8-f5c4-4cb2-85ef-0497e0e3a0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.131982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d69e8-f5c4-4cb2-85ef-0497e0e3a0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.132265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d69e8-f5c4-4cb2-85ef-0497e0e3a0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.132538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d69e8-f5c4-4cb2-85ef-0497e0e3a0a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.133257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d69e8-f5c4-4cb2-85ef-0497e0e3a0a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:19.133567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d69e8-f5c4-4cb2-85ef-0497e0e3a0a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.135045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d69e8-f5c4-4cb2-85ef-0497e0e3a0a5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.147648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 293/587] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:19.148461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d69e8-f5c4-4cb2-85ef-0497e0e3a0a5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.181038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496e1ac-56ab-46da-8d9e-2098e03d90f2 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.217201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a01d7c-4c37-46ed-a904-7d201c36ee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.217813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a01d7c-4c37-46ed-a904-7d201c36ee31 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.218195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a01d7c-4c37-46ed-a904-7d201c36ee31 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.218476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a01d7c-4c37-46ed-a904-7d201c36ee31 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.219121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a01d7c-4c37-46ed-a904-7d201c36ee31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:19.219429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a01d7c-4c37-46ed-a904-7d201c36ee31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.220968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a01d7c-4c37-46ed-a904-7d201c36ee31 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.238365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 295/588] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:19.238365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a01d7c-4c37-46ed-a904-7d201c36ee31 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.240999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:19.262070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d69e8-f5c4-4cb2-85ef-0497e0e3a0a5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.301586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2db9b-d69e-42ad-af70-bf95386bbd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.302730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2db9b-d69e-42ad-af70-bf95386bbd26 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.303013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2db9b-d69e-42ad-af70-bf95386bbd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.305098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2db9b-d69e-42ad-af70-bf95386bbd26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.318051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2db9b-d69e-42ad-af70-bf95386bbd26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.319538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2db9b-d69e-42ad-af70-bf95386bbd26 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-22b2db9b-d69e-42ad-af70-bf95386bbd26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:19.337001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:19.338058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:19.339057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:19.339057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:19.339057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:19.339057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 294/589] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:19.339057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2db9b-d69e-42ad-af70-bf95386bbd26 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.358767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a01d7c-4c37-46ed-a904-7d201c36ee31 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.385706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:19.395242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a11ed-9995-4192-92b2-cbb36abe935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.396156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a11ed-9995-4192-92b2-cbb36abe935e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.396437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a11ed-9995-4192-92b2-cbb36abe935e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.396709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a11ed-9995-4192-92b2-cbb36abe935e None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.397470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a11ed-9995-4192-92b2-cbb36abe935e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:19.397777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a11ed-9995-4192-92b2-cbb36abe935e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.399688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a11ed-9995-4192-92b2-cbb36abe935e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.418896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 296/590] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Sep 15 10:43:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:19.419803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a11ed-9995-4192-92b2-cbb36abe935e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.447942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2db9b-d69e-42ad-af70-bf95386bbd26 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.476860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:19.482024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31239ed0-c97e-4aa2-8958-1e6335c5afb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.482678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31239ed0-c97e-4aa2-8958-1e6335c5afb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.482958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31239ed0-c97e-4aa2-8958-1e6335c5afb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.483233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31239ed0-c97e-4aa2-8958-1e6335c5afb7 None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/218a015dd2b34435a76ecc8a763562ed/roles/fa23cf1cb845457d9460c2a9884af4cf` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.497714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31239ed0-c97e-4aa2-8958-1e6335c5afb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=218a015dd2b34435a76ecc8a763562ed, role_id=fa23cf1cb845457d9460c2a9884af4cf)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.499716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31239ed0-c97e-4aa2-8958-1e6335c5afb7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.511711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 295/591] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:43:19 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/218a015dd2b34435a76ecc8a763562ed/roles/fa23cf1cb845457d9460c2a9884af4cf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:43:19.512528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31239ed0-c97e-4aa2-8958-1e6335c5afb7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.520129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a11ed-9995-4192-92b2-cbb36abe935e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.566419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1c71c-fafd-4eba-8b2a-524dbc7875e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.567223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1c71c-fafd-4eba-8b2a-524dbc7875e3 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.567651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1c71c-fafd-4eba-8b2a-524dbc7875e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.567930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1c71c-fafd-4eba-8b2a-524dbc7875e3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.576293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1c71c-fafd-4eba-8b2a-524dbc7875e3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.578991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1c71c-fafd-4eba-8b2a-524dbc7875e3 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-38d1c71c-fafd-4eba-8b2a-524dbc7875e3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:19.591870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:19.593021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 297/592] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68248b47-23ab-4828-a7e0-e27523fe4d36 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.599306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68248b47-23ab-4828-a7e0-e27523fe4d36 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.600088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68248b47-23ab-4828-a7e0-e27523fe4d36 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.600088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 298/593] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:19.600088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68248b47-23ab-4828-a7e0-e27523fe4d36 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.630541 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:19.644362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31239ed0-c97e-4aa2-8958-1e6335c5afb7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.685144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee279cf-d7e2-4937-af0a-c155411a8985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.686341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee279cf-d7e2-4937-af0a-c155411a8985 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.686341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee279cf-d7e2-4937-af0a-c155411a8985 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.686341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee279cf-d7e2-4937-af0a-c155411a8985 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.687682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee279cf-d7e2-4937-af0a-c155411a8985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:19.687682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee279cf-d7e2-4937-af0a-c155411a8985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.688547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee279cf-d7e2-4937-af0a-c155411a8985 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.702653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 296/594] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:19.703737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee279cf-d7e2-4937-af0a-c155411a8985 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.722712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68248b47-23ab-4828-a7e0-e27523fe4d36 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.744070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:19.759912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac633796-764e-407f-bfce-658c2d619299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.760550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac633796-764e-407f-bfce-658c2d619299 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.760928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac633796-764e-407f-bfce-658c2d619299 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.761349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac633796-764e-407f-bfce-658c2d619299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.773768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac633796-764e-407f-bfce-658c2d619299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.775230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac633796-764e-407f-bfce-658c2d619299 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-ac633796-764e-407f-bfce-658c2d619299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:19.789430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:19.790631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:19.791714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:19.791714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:19.791714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:19.791714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 299/595] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:19.791994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd709e-e337-421b-ab2c-24fb673bf987 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.792281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd709e-e337-421b-ab2c-24fb673bf987 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.792554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd709e-e337-421b-ab2c-24fb673bf987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.811532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-42cd709e-e337-421b-ab2c-24fb673bf987 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:19.813309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-42cd709e-e337-421b-ab2c-24fb673bf987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:19.826697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee279cf-d7e2-4937-af0a-c155411a8985 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:19.865794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40d883-f7b1-4770-92a9-997216a8326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:19.866330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40d883-f7b1-4770-92a9-997216a8326d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:19.866681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40d883-f7b1-4770-92a9-997216a8326d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:19.867081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40d883-f7b1-4770-92a9-997216a8326d None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:19.867814 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40d883-f7b1-4770-92a9-997216a8326d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:19.868191 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40d883-f7b1-4770-92a9-997216a8326d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:19.869732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40d883-f7b1-4770-92a9-997216a8326d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:19.892788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 300/596] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:19.893009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 297/597] 10.222.0.50 () {56 vars in 1139 bytes} [Thu Sep 15 10:43:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:19.893009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40d883-f7b1-4770-92a9-997216a8326d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.895553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd709e-e337-421b-ab2c-24fb673bf987 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:19.909209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:20.005698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40d883-f7b1-4770-92a9-997216a8326d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.008472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd709e-e337-421b-ab2c-24fb673bf987 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.010118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:20.043408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977acc99-11e2-4b92-add2-6113bfa25c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.044289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977acc99-11e2-4b92-add2-6113bfa25c9a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.044750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977acc99-11e2-4b92-add2-6113bfa25c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.045056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977acc99-11e2-4b92-add2-6113bfa25c9a None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.045795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977acc99-11e2-4b92-add2-6113bfa25c9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:20.046733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977acc99-11e2-4b92-add2-6113bfa25c9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.048397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccaad1-6af2-42b2-8bbe-a5ff90ff8e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.048526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977acc99-11e2-4b92-add2-6113bfa25c9a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.049401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccaad1-6af2-42b2-8bbe-a5ff90ff8e09 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.050004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccaad1-6af2-42b2-8bbe-a5ff90ff8e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.050494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccaad1-6af2-42b2-8bbe-a5ff90ff8e09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.058149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccaad1-6af2-42b2-8bbe-a5ff90ff8e09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.059920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccaad1-6af2-42b2-8bbe-a5ff90ff8e09 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.064610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 298/598] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:20.064610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977acc99-11e2-4b92-add2-6113bfa25c9a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-33ccaad1-6af2-42b2-8bbe-a5ff90ff8e09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:20.078768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:20.079702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:20.080617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:20.080617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:20.080617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:20.080617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:20.080617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:20.080617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:20.080617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:20.080617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:20.083360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 301/599] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:20.084360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccaad1-6af2-42b2-8bbe-a5ff90ff8e09 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.134295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:20.172712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977acc99-11e2-4b92-add2-6113bfa25c9a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.197670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccaad1-6af2-42b2-8bbe-a5ff90ff8e09 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.209726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7dbae-153c-4414-81d7-2f9d61e6b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.210310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7dbae-153c-4414-81d7-2f9d61e6b15b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.210618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7dbae-153c-4414-81d7-2f9d61e6b15b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.210888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7dbae-153c-4414-81d7-2f9d61e6b15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.223939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7dbae-153c-4414-81d7-2f9d61e6b15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.225730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7dbae-153c-4414-81d7-2f9d61e6b15b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.233259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c660466-cfe1-4c92-9d8e-c55f8f8362dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.234677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c660466-cfe1-4c92-9d8e-c55f8f8362dd None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.234953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c660466-cfe1-4c92-9d8e-c55f8f8362dd None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.235220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c660466-cfe1-4c92-9d8e-c55f8f8362dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-9bd7dbae-153c-4414-81d7-2f9d61e6b15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:20.241523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:20.242499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:20.247921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:20.247921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:20.247921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:20.248097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c660466-cfe1-4c92-9d8e-c55f8f8362dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.248183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 299/600] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:20.248255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c660466-cfe1-4c92-9d8e-c55f8f8362dd None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.248332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7dbae-153c-4414-81d7-2f9d61e6b15b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-8c660466-cfe1-4c92-9d8e-c55f8f8362dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:20.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:20.261302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:20.262448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:20.262448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:20.262448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:20.262448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:20.262448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:20.262448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:20.262448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:20.262448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:20.262984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 302/601] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:20.267802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c660466-cfe1-4c92-9d8e-c55f8f8362dd None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.290444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:20.357243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7dbae-153c-4414-81d7-2f9d61e6b15b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.373428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c660466-cfe1-4c92-9d8e-c55f8f8362dd None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.378509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:20.394033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e789a-14dd-43a3-bfec-455ba7d2e788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.394645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e789a-14dd-43a3-bfec-455ba7d2e788 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.395052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e789a-14dd-43a3-bfec-455ba7d2e788 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.395330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e789a-14dd-43a3-bfec-455ba7d2e788 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.396022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e789a-14dd-43a3-bfec-455ba7d2e788 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:20.396323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e789a-14dd-43a3-bfec-455ba7d2e788 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.397896 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e789a-14dd-43a3-bfec-455ba7d2e788 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.410314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f43fd0-4b06-4fa2-898b-9d043439339d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.416192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 300/602] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:20.416391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f43fd0-4b06-4fa2-898b-9d043439339d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.416391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f43fd0-4b06-4fa2-898b-9d043439339d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.416391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f43fd0-4b06-4fa2-898b-9d043439339d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.416662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e789a-14dd-43a3-bfec-455ba7d2e788 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.428469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f43fd0-4b06-4fa2-898b-9d043439339d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-f8f43fd0-4b06-4fa2-898b-9d043439339d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:20.434097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:20.435054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:20.436106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:20.440444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 303/603] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:20.441256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f43fd0-4b06-4fa2-898b-9d043439339d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.525871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:20.528198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e789a-14dd-43a3-bfec-455ba7d2e788 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.549796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f43fd0-4b06-4fa2-898b-9d043439339d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.568599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5f1da-0367-42e0-952d-ca4a62acf93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.569523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5f1da-0367-42e0-952d-ca4a62acf93b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.569803 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5f1da-0367-42e0-952d-ca4a62acf93b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.570075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5f1da-0367-42e0-952d-ca4a62acf93b None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.570710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5f1da-0367-42e0-952d-ca4a62acf93b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:20.571010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5f1da-0367-42e0-952d-ca4a62acf93b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.572830 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5f1da-0367-42e0-952d-ca4a62acf93b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.589974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 301/604] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:20.589974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5f1da-0367-42e0-952d-ca4a62acf93b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.590944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25e8ad-22e9-44fe-8eea-21c4f3f059ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.591618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25e8ad-22e9-44fe-8eea-21c4f3f059ba None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.591962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25e8ad-22e9-44fe-8eea-21c4f3f059ba None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.592314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25e8ad-22e9-44fe-8eea-21c4f3f059ba None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.593237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25e8ad-22e9-44fe-8eea-21c4f3f059ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:20.593605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25e8ad-22e9-44fe-8eea-21c4f3f059ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.595136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25e8ad-22e9-44fe-8eea-21c4f3f059ba None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.612196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 304/605] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:20.612196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25e8ad-22e9-44fe-8eea-21c4f3f059ba None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.639868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:20.703867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5f1da-0367-42e0-952d-ca4a62acf93b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.719510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25e8ad-22e9-44fe-8eea-21c4f3f059ba None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.739823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94140f4b-c925-4fa3-9278-a82a48976309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.740477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94140f4b-c925-4fa3-9278-a82a48976309 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.740753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94140f4b-c925-4fa3-9278-a82a48976309 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.741056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94140f4b-c925-4fa3-9278-a82a48976309 None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/6908a326a3d3437c9db898976a2def1b/roles/fa23cf1cb845457d9460c2a9884af4cf` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.756615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94140f4b-c925-4fa3-9278-a82a48976309 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=6908a326a3d3437c9db898976a2def1b, role_id=fa23cf1cb845457d9460c2a9884af4cf)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.758066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94140f4b-c925-4fa3-9278-a82a48976309 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.763543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234804df-36c3-4293-876a-470a5ef6d922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.764276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234804df-36c3-4293-876a-470a5ef6d922 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.764690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234804df-36c3-4293-876a-470a5ef6d922 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.765333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234804df-36c3-4293-876a-470a5ef6d922 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.766066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234804df-36c3-4293-876a-470a5ef6d922 None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:20.766474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234804df-36c3-4293-876a-470a5ef6d922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.767954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234804df-36c3-4293-876a-470a5ef6d922 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.781288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 302/606] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:43:20 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/6908a326a3d3437c9db898976a2def1b/roles/fa23cf1cb845457d9460c2a9884af4cf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:43:20.781288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94140f4b-c925-4fa3-9278-a82a48976309 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.786070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 305/607] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:20.787046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234804df-36c3-4293-876a-470a5ef6d922 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:20.790871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:20.912179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234804df-36c3-4293-876a-470a5ef6d922 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.914207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:20.929439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94140f4b-c925-4fa3-9278-a82a48976309 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:20.962300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d099df-f5ff-4ff0-b7fd-ceb62bc6ad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.962939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d099df-f5ff-4ff0-b7fd-ceb62bc6ad8f None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.963361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d099df-f5ff-4ff0-b7fd-ceb62bc6ad8f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.963700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d099df-f5ff-4ff0-b7fd-ceb62bc6ad8f None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/d7fa5d550a2d442a820cf4e12ebc36c5/roles/fa23cf1cb845457d9460c2a9884af4cf` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.974114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0aa63-3b75-4dd0-95eb-e25d98cdda5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:20.974756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0aa63-3b75-4dd0-95eb-e25d98cdda5d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:20.975039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0aa63-3b75-4dd0-95eb-e25d98cdda5d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:20.975307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0aa63-3b75-4dd0-95eb-e25d98cdda5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:20.983371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d099df-f5ff-4ff0-b7fd-ceb62bc6ad8f None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=d7fa5d550a2d442a820cf4e12ebc36c5, role_id=fa23cf1cb845457d9460c2a9884af4cf)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.986230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d099df-f5ff-4ff0-b7fd-ceb62bc6ad8f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:20.988203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0aa63-3b75-4dd0-95eb-e25d98cdda5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:20.989704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0aa63-3b75-4dd0-95eb-e25d98cdda5d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:21.003860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 306/608] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:43:20 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/d7fa5d550a2d442a820cf4e12ebc36c5/roles/fa23cf1cb845457d9460c2a9884af4cf => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-cad0aa63-3b75-4dd0-95eb-e25d98cdda5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:21.004067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:21.005035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:21.005983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:21.005983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:21.005983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:21.005983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:21.005983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:21.005983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:21.005983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:21.005983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:21.006410 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d099df-f5ff-4ff0-b7fd-ceb62bc6ad8f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:21.006491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 303/609] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:21.011248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0aa63-3b75-4dd0-95eb-e25d98cdda5d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:21.096116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:21.139277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0aa63-3b75-4dd0-95eb-e25d98cdda5d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:21.167707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d099df-f5ff-4ff0-b7fd-ceb62bc6ad8f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:21.178228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160e83a-b352-42c7-a245-57774eec33b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:21.178839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160e83a-b352-42c7-a245-57774eec33b8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:21.179115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160e83a-b352-42c7-a245-57774eec33b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:21.179385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160e83a-b352-42c7-a245-57774eec33b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:21.180081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160e83a-b352-42c7-a245-57774eec33b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:21.180382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160e83a-b352-42c7-a245-57774eec33b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:21.181845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160e83a-b352-42c7-a245-57774eec33b8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:21.201103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 304/610] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:21.202886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160e83a-b352-42c7-a245-57774eec33b8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:21.210728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ac9d3-3cfe-4bd4-bef8-840184bde491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:21.211370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ac9d3-3cfe-4bd4-bef8-840184bde491 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:21.211646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ac9d3-3cfe-4bd4-bef8-840184bde491 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:21.211913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ac9d3-3cfe-4bd4-bef8-840184bde491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:21.216821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:21.228244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ac9d3-3cfe-4bd4-bef8-840184bde491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:21.230249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ac9d3-3cfe-4bd4-bef8-840184bde491 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-7d7ac9d3-3cfe-4bd4-bef8-840184bde491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:21.253715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:21.254743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:21.255662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:21.255662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:21.255662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:21.257556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 307/611] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:21.263159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ac9d3-3cfe-4bd4-bef8-840184bde491 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:21.325374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160e83a-b352-42c7-a245-57774eec33b8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:21.361049 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcee86-2749-4e70-b83a-31159e91b50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:21.361806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcee86-2749-4e70-b83a-31159e91b50a None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:21.362195 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcee86-2749-4e70-b83a-31159e91b50a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:21.362528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcee86-2749-4e70-b83a-31159e91b50a None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/f5e7b08dc3544e76ac6732ca2aa89bf5/roles/fa23cf1cb845457d9460c2a9884af4cf` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:21.379847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcee86-2749-4e70-b83a-31159e91b50a None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=f5e7b08dc3544e76ac6732ca2aa89bf5, role_id=fa23cf1cb845457d9460c2a9884af4cf)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:21.381593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcee86-2749-4e70-b83a-31159e91b50a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:21.384405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ac9d3-3cfe-4bd4-bef8-840184bde491 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:21.393495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:21.401032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 305/612] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:43:21 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/f5e7b08dc3544e76ac6732ca2aa89bf5/roles/fa23cf1cb845457d9460c2a9884af4cf => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:43:21.425753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518dde0-88ba-4d2a-ad1a-36b94a827b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:21.426520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518dde0-88ba-4d2a-ad1a-36b94a827b54 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:21.426871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518dde0-88ba-4d2a-ad1a-36b94a827b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:21.427210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518dde0-88ba-4d2a-ad1a-36b94a827b54 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:21.427936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518dde0-88ba-4d2a-ad1a-36b94a827b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:21.428246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518dde0-88ba-4d2a-ad1a-36b94a827b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:21.429678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518dde0-88ba-4d2a-ad1a-36b94a827b54 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:21.444205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 308/613] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:21.449885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcee86-2749-4e70-b83a-31159e91b50a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:21.530677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:21.606714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcee86-2749-4e70-b83a-31159e91b50a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:21.660905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616670f4-0f66-4cfb-8d20-a30aa16e0b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:21.661613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616670f4-0f66-4cfb-8d20-a30aa16e0b29 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:21.661846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616670f4-0f66-4cfb-8d20-a30aa16e0b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:21.662115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616670f4-0f66-4cfb-8d20-a30aa16e0b29 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:21.678982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616670f4-0f66-4cfb-8d20-a30aa16e0b29 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:21.681019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616670f4-0f66-4cfb-8d20-a30aa16e0b29 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-616670f4-0f66-4cfb-8d20-a30aa16e0b29 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:21.694841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:21.697112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:21.699362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 306/614] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:21.702846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518dde0-88ba-4d2a-ad1a-36b94a827b54 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:21.783712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e20cbd-b6a4-408d-bb64-58000cb2d37c None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:21.784013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e20cbd-b6a4-408d-bb64-58000cb2d37c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:21.784181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e20cbd-b6a4-408d-bb64-58000cb2d37c None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:21.785961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 307/615] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:21.793678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b896e-dfb9-4e1f-9c5e-5bca75c579e3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:21.793952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b896e-dfb9-4e1f-9c5e-5bca75c579e3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:21.794227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b896e-dfb9-4e1f-9c5e-5bca75c579e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:21.822197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-482b896e-dfb9-4e1f-9c5e-5bca75c579e3 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:21.822424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518dde0-88ba-4d2a-ad1a-36b94a827b54 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:21.824226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-482b896e-dfb9-4e1f-9c5e-5bca75c579e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:21.860919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3031bf6-ff37-4c57-936e-b7b64d458251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:21.861528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3031bf6-ff37-4c57-936e-b7b64d458251 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:21.861805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3031bf6-ff37-4c57-936e-b7b64d458251 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:21.862074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3031bf6-ff37-4c57-936e-b7b64d458251 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:21.862798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3031bf6-ff37-4c57-936e-b7b64d458251 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:21.863099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3031bf6-ff37-4c57-936e-b7b64d458251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:21.864535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3031bf6-ff37-4c57-936e-b7b64d458251 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:21.883840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 309/616] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Sep 15 10:43:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:21.889310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3031bf6-ff37-4c57-936e-b7b64d458251 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:21.920331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:21.987872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 308/617] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:43:22.025841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3031bf6-ff37-4c57-936e-b7b64d458251 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:22.082479 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02efebc-59f1-4f30-b53d-48e45e3c7bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:22.083359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02efebc-59f1-4f30-b53d-48e45e3c7bef None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:22.083794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02efebc-59f1-4f30-b53d-48e45e3c7bef None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:22.084207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02efebc-59f1-4f30-b53d-48e45e3c7bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:22.100251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02efebc-59f1-4f30-b53d-48e45e3c7bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:22.101907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02efebc-59f1-4f30-b53d-48e45e3c7bef None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-e02efebc-59f1-4f30-b53d-48e45e3c7bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:22.122038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:22.123230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:22.124708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:22.124708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:22.124708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:22.126533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 310/618] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:22.132373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b896e-dfb9-4e1f-9c5e-5bca75c579e3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:22.195704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:22.268603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b896e-dfb9-4e1f-9c5e-5bca75c579e3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:22.306620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471a80c-e9a8-4a5a-a470-9ca8f1c9f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:22.307223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471a80c-e9a8-4a5a-a470-9ca8f1c9f170 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:22.307499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471a80c-e9a8-4a5a-a470-9ca8f1c9f170 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:22.307768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471a80c-e9a8-4a5a-a470-9ca8f1c9f170 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:22.308405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471a80c-e9a8-4a5a-a470-9ca8f1c9f170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:22.308700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471a80c-e9a8-4a5a-a470-9ca8f1c9f170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:22.310113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471a80c-e9a8-4a5a-a470-9ca8f1c9f170 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:22.322539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 309/619] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:22.329140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02efebc-59f1-4f30-b53d-48e45e3c7bef None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:22.388575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471a80c-e9a8-4a5a-a470-9ca8f1c9f170 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:22.430053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:22.476443 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02efebc-59f1-4f30-b53d-48e45e3c7bef None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:22.552214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fca3fc-8f44-406a-b35c-78123ed21462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:22.552874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fca3fc-8f44-406a-b35c-78123ed21462 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:22.553139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fca3fc-8f44-406a-b35c-78123ed21462 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:22.557392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fca3fc-8f44-406a-b35c-78123ed21462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:22.569428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fca3fc-8f44-406a-b35c-78123ed21462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:22.570886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fca3fc-8f44-406a-b35c-78123ed21462 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:22.585323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471a80c-e9a8-4a5a-a470-9ca8f1c9f170 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-b7fca3fc-8f44-406a-b35c-78123ed21462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:22.599042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:22.599983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:22.601104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:22.601104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:22.601104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:22.601104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:22.601104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:22.601104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:22.601104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:22.601104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:22.604917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 311/620] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:22.606607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:22.607313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fca3fc-8f44-406a-b35c-78123ed21462 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:22.656144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c198b-2abd-4fa8-82ad-f3dd7fd9c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:22.656778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c198b-2abd-4fa8-82ad-f3dd7fd9c063 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:22.657079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c198b-2abd-4fa8-82ad-f3dd7fd9c063 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:22.657347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c198b-2abd-4fa8-82ad-f3dd7fd9c063 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:22.678354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c198b-2abd-4fa8-82ad-f3dd7fd9c063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c44c198b-2abd-4fa8-82ad-f3dd7fd9c063 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:22.683612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:22.684774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:22.686005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 310/621] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:22.691045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c198b-2abd-4fa8-82ad-f3dd7fd9c063 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:22.785107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fca3fc-8f44-406a-b35c-78123ed21462 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:22.843545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6e144-0d58-4a51-9090-5f9f443de0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:22.848145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6e144-0d58-4a51-9090-5f9f443de0cf None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:22.848677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6e144-0d58-4a51-9090-5f9f443de0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:22.849179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6e144-0d58-4a51-9090-5f9f443de0cf None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:22.850083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6e144-0d58-4a51-9090-5f9f443de0cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:22.850779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6e144-0d58-4a51-9090-5f9f443de0cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:22.853135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6e144-0d58-4a51-9090-5f9f443de0cf None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:22.856115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:22.871849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 312/622] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:22.879149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6e144-0d58-4a51-9090-5f9f443de0cf None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:22.907944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c198b-2abd-4fa8-82ad-f3dd7fd9c063 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:22.967911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f8cbd-9f2b-46ca-8314-156f3b48bd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:22.968765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f8cbd-9f2b-46ca-8314-156f3b48bd3e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:22.969071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f8cbd-9f2b-46ca-8314-156f3b48bd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:22.969343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f8cbd-9f2b-46ca-8314-156f3b48bd3e None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:22.970020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f8cbd-9f2b-46ca-8314-156f3b48bd3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:22.970325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f8cbd-9f2b-46ca-8314-156f3b48bd3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:22.971724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f8cbd-9f2b-46ca-8314-156f3b48bd3e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:22.988008 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 311/623] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:22.993731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f8cbd-9f2b-46ca-8314-156f3b48bd3e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:23.033199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:23.107668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f8cbd-9f2b-46ca-8314-156f3b48bd3e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:23.111143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6e144-0d58-4a51-9090-5f9f443de0cf None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:23.165372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f465a-a2da-4165-9928-056cf3f140bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:23.165924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f465a-a2da-4165-9928-056cf3f140bc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:23.166206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f465a-a2da-4165-9928-056cf3f140bc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:23.166479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f465a-a2da-4165-9928-056cf3f140bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:23.179060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85677ff5-d2ed-421f-9eda-386c92c5a513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:23.179720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85677ff5-d2ed-421f-9eda-386c92c5a513 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:23.179999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85677ff5-d2ed-421f-9eda-386c92c5a513 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:23.180273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85677ff5-d2ed-421f-9eda-386c92c5a513 None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/06007b3b2260409f863ac3d0ac85f4cc/roles/fa23cf1cb845457d9460c2a9884af4cf` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:23.185370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f465a-a2da-4165-9928-056cf3f140bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:23.187176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f465a-a2da-4165-9928-056cf3f140bc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:23.210125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85677ff5-d2ed-421f-9eda-386c92c5a513 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=06007b3b2260409f863ac3d0ac85f4cc, role_id=fa23cf1cb845457d9460c2a9884af4cf)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-440f465a-a2da-4165-9928-056cf3f140bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:23.211665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:23.212643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:23.213587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:23.213587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:23.213587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:23.214607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85677ff5-d2ed-421f-9eda-386c92c5a513 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:23.217153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 312/624] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:23.219322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f465a-a2da-4165-9928-056cf3f140bc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:23.236851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 313/625] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:43:22 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/06007b3b2260409f863ac3d0ac85f4cc/roles/fa23cf1cb845457d9460c2a9884af4cf => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:43:23.349334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:23.377348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f465a-a2da-4165-9928-056cf3f140bc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:23.425249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5fd0e-9546-4d3a-8c28-ac525576bfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:23.425797 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fd0e-9546-4d3a-8c28-ac525576bfaf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:23.426079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fd0e-9546-4d3a-8c28-ac525576bfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:23.426354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fd0e-9546-4d3a-8c28-ac525576bfaf None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:23.427008 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5fd0e-9546-4d3a-8c28-ac525576bfaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:23.427310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5fd0e-9546-4d3a-8c28-ac525576bfaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:23.428712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5fd0e-9546-4d3a-8c28-ac525576bfaf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:23.445157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 313/626] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:23.450834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85677ff5-d2ed-421f-9eda-386c92c5a513 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:23.464406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70a104-21cc-47b0-ab98-dcaab04ca3a0 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:23.465021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70a104-21cc-47b0-ab98-dcaab04ca3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:23.465484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70a104-21cc-47b0-ab98-dcaab04ca3a0 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:23.467242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 314/627] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:43:23.473592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f24d4-6ede-42ba-9b01-5f41d5353aa1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:23.474117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f24d4-6ede-42ba-9b01-5f41d5353aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:23.475026 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f24d4-6ede-42ba-9b01-5f41d5353aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:23.503291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4a0f24d4-6ede-42ba-9b01-5f41d5353aa1 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:23.504781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4a0f24d4-6ede-42ba-9b01-5f41d5353aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:23.516695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:23.590934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85677ff5-d2ed-421f-9eda-386c92c5a513 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:23.598087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 315/628] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:43:23.629017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e87fd-a895-477f-b7e5-af6497aab514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:23.629922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e87fd-a895-477f-b7e5-af6497aab514 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:23.630351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e87fd-a895-477f-b7e5-af6497aab514 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:23.630859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e87fd-a895-477f-b7e5-af6497aab514 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:23.638445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e87fd-a895-477f-b7e5-af6497aab514 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:23.640379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e87fd-a895-477f-b7e5-af6497aab514 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-ab6e87fd-a895-477f-b7e5-af6497aab514 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:23.653204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:23.654497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:23.655687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:23.657875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 314/629] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Sep 15 10:43:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:23.663612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f24d4-6ede-42ba-9b01-5f41d5353aa1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:23.746266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:23.774130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f24d4-6ede-42ba-9b01-5f41d5353aa1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:23.810589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34eb76-bee8-486a-b218-fda6935b0cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:23.811491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34eb76-bee8-486a-b218-fda6935b0cea None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:23.812007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34eb76-bee8-486a-b218-fda6935b0cea None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:23.812427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34eb76-bee8-486a-b218-fda6935b0cea None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:23.813344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34eb76-bee8-486a-b218-fda6935b0cea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:23.813809 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34eb76-bee8-486a-b218-fda6935b0cea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:23.815379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34eb76-bee8-486a-b218-fda6935b0cea None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:23.835775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 316/630] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Sep 15 10:43:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:23.841595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e87fd-a895-477f-b7e5-af6497aab514 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:23.947500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e87fd-a895-477f-b7e5-af6497aab514 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:23.956578 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:23.985238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4ef4e-bc41-4535-b29b-bb76f02041a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:23.985847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4ef4e-bc41-4535-b29b-bb76f02041a7 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:23.986130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4ef4e-bc41-4535-b29b-bb76f02041a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:23.986402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4ef4e-bc41-4535-b29b-bb76f02041a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.026356 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34eb76-bee8-486a-b218-fda6935b0cea None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:24.059678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4ef4e-bc41-4535-b29b-bb76f02041a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:24.061184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4ef4e-bc41-4535-b29b-bb76f02041a7 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:24.235208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34eb76-bee8-486a-b218-fda6935b0cea None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:24.238290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 315/631] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:24.239628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca091b0-c459-43bd-893e-12571eeecc7a None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.239916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca091b0-c459-43bd-893e-12571eeecc7a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.240193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca091b0-c459-43bd-893e-12571eeecc7a None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.246312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 316/632] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:24.247116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca091b0-c459-43bd-893e-12571eeecc7a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:24.257254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:24.269046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:24.275578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120ecd5-bbb7-430b-a00e-68fdab22180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:24.276216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120ecd5-bbb7-430b-a00e-68fdab22180c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.276548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120ecd5-bbb7-430b-a00e-68fdab22180c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.276769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120ecd5-bbb7-430b-a00e-68fdab22180c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.288496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120ecd5-bbb7-430b-a00e-68fdab22180c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c120ecd5-bbb7-430b-a00e-68fdab22180c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:24.293489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:24.294437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:24.295291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:24.300099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 317/633] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:24.301183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a3d8-8e2e-4bee-92b3-553751161872 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.301472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a3d8-8e2e-4bee-92b3-553751161872 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.301751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a3d8-8e2e-4bee-92b3-553751161872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.320486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ec11a3d8-8e2e-4bee-92b3-553751161872 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:24.321713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ec11a3d8-8e2e-4bee-92b3-553751161872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:24.360862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca091b0-c459-43bd-893e-12571eeecc7a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:24.397418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5e1ca-be55-4356-9b6a-19e50b5f897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:24.398004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 318/634] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:24.398133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e1ca-be55-4356-9b6a-19e50b5f897a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.398422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e1ca-be55-4356-9b6a-19e50b5f897a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.398678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e1ca-be55-4356-9b6a-19e50b5f897a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.399252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa1e81-d1e6-433a-b26f-53b4cd96165f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.399488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa1e81-d1e6-433a-b26f-53b4cd96165f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.399775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa1e81-d1e6-433a-b26f-53b4cd96165f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.405528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 319/635] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:24.405528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa1e81-d1e6-433a-b26f-53b4cd96165f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:24.419307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5e1ca-be55-4356-9b6a-19e50b5f897a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:24.422195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5e1ca-be55-4356-9b6a-19e50b5f897a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-37d5e1ca-be55-4356-9b6a-19e50b5f897a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:24.442843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:24.443872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:24.444795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:24.444795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:24.444795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:24.452127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 317/636] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:24.452127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9344b3c-0ad3-4bfa-bd0e-d9feb2911f72 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.452127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9344b3c-0ad3-4bfa-bd0e-d9feb2911f72 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.452127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9344b3c-0ad3-4bfa-bd0e-d9feb2911f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.477097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-a9344b3c-0ad3-4bfa-bd0e-d9feb2911f72 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:24.479000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a9344b3c-0ad3-4bfa-bd0e-d9feb2911f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:24.520587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa1e81-d1e6-433a-b26f-53b4cd96165f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:24.548318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:24.551285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:24.566839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e34036-99c5-4d32-8615-76660e84798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:24.567432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e34036-99c5-4d32-8615-76660e84798a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.567713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e34036-99c5-4d32-8615-76660e84798a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.567997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e34036-99c5-4d32-8615-76660e84798a None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.568758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e34036-99c5-4d32-8615-76660e84798a None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:24.569015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e34036-99c5-4d32-8615-76660e84798a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:24.570412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e34036-99c5-4d32-8615-76660e84798a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:24.574014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 318/637] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:24.575050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9344b3c-0ad3-4bfa-bd0e-d9feb2911f72 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:24.588231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 320/638] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:24.597738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e34036-99c5-4d32-8615-76660e84798a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:24.691363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9344b3c-0ad3-4bfa-bd0e-d9feb2911f72 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:24.734860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed03046-ebfd-427e-8a18-913a64fd6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:24.735775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed03046-ebfd-427e-8a18-913a64fd6de4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.736262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed03046-ebfd-427e-8a18-913a64fd6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.736714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed03046-ebfd-427e-8a18-913a64fd6de4 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.737548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed03046-ebfd-427e-8a18-913a64fd6de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:24.738029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed03046-ebfd-427e-8a18-913a64fd6de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:24.739251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e34036-99c5-4d32-8615-76660e84798a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:24.739824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed03046-ebfd-427e-8a18-913a64fd6de4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:24.754411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 319/639] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:24.760330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed03046-ebfd-427e-8a18-913a64fd6de4 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:24.778108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae469829-7f52-480a-b72e-5fcf85b19b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:24.778927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469829-7f52-480a-b72e-5fcf85b19b22 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.779218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469829-7f52-480a-b72e-5fcf85b19b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.779494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469829-7f52-480a-b72e-5fcf85b19b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.790612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae469829-7f52-480a-b72e-5fcf85b19b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:24.792080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae469829-7f52-480a-b72e-5fcf85b19b22 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ae469829-7f52-480a-b72e-5fcf85b19b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:24.808206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:24.809249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:24.810284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:24.810284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:24.810284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:24.810284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 321/640] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:24.810965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae469829-7f52-480a-b72e-5fcf85b19b22 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:24.824488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:24.850158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:24.888004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed03046-ebfd-427e-8a18-913a64fd6de4 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:24.920052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae469829-7f52-480a-b72e-5fcf85b19b22 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:24.925739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fd99-b450-42a4-ad14-02740ccc6291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:24.926371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fd99-b450-42a4-ad14-02740ccc6291 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.926652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fd99-b450-42a4-ad14-02740ccc6291 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.926924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fd99-b450-42a4-ad14-02740ccc6291 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.939203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483fd99-b450-42a4-ad14-02740ccc6291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-e483fd99-b450-42a4-ad14-02740ccc6291 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:24.945618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:24.946607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:24.947492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 320/641] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:24.948643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fd99-b450-42a4-ad14-02740ccc6291 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:24.961085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6502c1f-6ec9-45cc-8393-b1a797a8c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:24.961724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502c1f-6ec9-45cc-8393-b1a797a8c8fa None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:24.962010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502c1f-6ec9-45cc-8393-b1a797a8c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:24.962286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502c1f-6ec9-45cc-8393-b1a797a8c8fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:24.969740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6502c1f-6ec9-45cc-8393-b1a797a8c8fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:24.971193 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6502c1f-6ec9-45cc-8393-b1a797a8c8fa None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e6502c1f-6ec9-45cc-8393-b1a797a8c8fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:24.982661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:24.984211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:24.985223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:24.985223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:24.985223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:24.985223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:24.985223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:24.985223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:24.985223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:24.985223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:24.989053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 322/642] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:24.989965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6502c1f-6ec9-45cc-8393-b1a797a8c8fa None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.060359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fd99-b450-42a4-ad14-02740ccc6291 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.071303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:25.089863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:25.097507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6502c1f-6ec9-45cc-8393-b1a797a8c8fa None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.101220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2ce38-915b-463c-9120-43250d28b317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.101828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2ce38-915b-463c-9120-43250d28b317 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.102118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2ce38-915b-463c-9120-43250d28b317 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.102394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2ce38-915b-463c-9120-43250d28b317 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.103040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2ce38-915b-463c-9120-43250d28b317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:25.103346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2ce38-915b-463c-9120-43250d28b317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:25.104713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2ce38-915b-463c-9120-43250d28b317 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:25.121174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 321/643] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:25.121174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2ce38-915b-463c-9120-43250d28b317 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.136793 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ac9fe-f14d-45f9-b3cc-436562e00166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.137588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ac9fe-f14d-45f9-b3cc-436562e00166 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.137869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ac9fe-f14d-45f9-b3cc-436562e00166 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.138130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ac9fe-f14d-45f9-b3cc-436562e00166 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.149510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ac9fe-f14d-45f9-b3cc-436562e00166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-fa5ac9fe-f14d-45f9-b3cc-436562e00166 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:25.155760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:25.156711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:25.157672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:25.159189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 323/644] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:25.159959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ac9fe-f14d-45f9-b3cc-436562e00166 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.231930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2ce38-915b-463c-9120-43250d28b317 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.270076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f980e2-014d-4693-8a11-e2f9b40f8f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.270685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f980e2-014d-4693-8a11-e2f9b40f8f97 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.270967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f980e2-014d-4693-8a11-e2f9b40f8f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.271245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f980e2-014d-4693-8a11-e2f9b40f8f97 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.271726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ac9fe-f14d-45f9-b3cc-436562e00166 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.271908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f980e2-014d-4693-8a11-e2f9b40f8f97 None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:25.272204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f980e2-014d-4693-8a11-e2f9b40f8f97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:25.273612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f980e2-014d-4693-8a11-e2f9b40f8f97 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:25.283007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 322/645] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:25.283820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f980e2-014d-4693-8a11-e2f9b40f8f97 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.315506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed84f2-1581-47a1-bd87-b0105f5924f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.316401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed84f2-1581-47a1-bd87-b0105f5924f2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.316897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed84f2-1581-47a1-bd87-b0105f5924f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.317359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed84f2-1581-47a1-bd87-b0105f5924f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.318208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:25.318343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed84f2-1581-47a1-bd87-b0105f5924f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:25.318557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed84f2-1581-47a1-bd87-b0105f5924f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:25.320069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed84f2-1581-47a1-bd87-b0105f5924f2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:25.338494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 324/646] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:25.339483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed84f2-1581-47a1-bd87-b0105f5924f2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.362029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:25.394097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f980e2-014d-4693-8a11-e2f9b40f8f97 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.428517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaae07f-ee94-4898-a957-ebe92eeadbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.429177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaae07f-ee94-4898-a957-ebe92eeadbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.429460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaae07f-ee94-4898-a957-ebe92eeadbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.429732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaae07f-ee94-4898-a957-ebe92eeadbf4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.437371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaae07f-ee94-4898-a957-ebe92eeadbf4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:25.438816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaae07f-ee94-4898-a957-ebe92eeadbf4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-ffaae07f-ee94-4898-a957-ebe92eeadbf4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:25.449930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:25.450905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 323/647] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:25.457170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaae07f-ee94-4898-a957-ebe92eeadbf4 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.462124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed84f2-1581-47a1-bd87-b0105f5924f2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.500026 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0d9c-b94f-4699-b362-7e22278b3baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.500646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0d9c-b94f-4699-b362-7e22278b3baa None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.500968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0d9c-b94f-4699-b362-7e22278b3baa None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.501247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0d9c-b94f-4699-b362-7e22278b3baa None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.501914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0d9c-b94f-4699-b362-7e22278b3baa None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:25.502257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0d9c-b94f-4699-b362-7e22278b3baa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:25.503651 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0d9c-b94f-4699-b362-7e22278b3baa None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:25.511503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 325/648] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:25.512290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0d9c-b94f-4699-b362-7e22278b3baa None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.571097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:25.584466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaae07f-ee94-4898-a957-ebe92eeadbf4 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.607612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:25.620834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0746c3-1480-4489-890b-9bc602bffab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.622002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0746c3-1480-4489-890b-9bc602bffab6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.622291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0746c3-1480-4489-890b-9bc602bffab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.622723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0746c3-1480-4489-890b-9bc602bffab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.636026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0746c3-1480-4489-890b-9bc602bffab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:25.636400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0d9c-b94f-4699-b362-7e22278b3baa None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.637584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0746c3-1480-4489-890b-9bc602bffab6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-5a0746c3-1480-4489-890b-9bc602bffab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:25.659860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:25.660952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:25.661874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:25.661874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:25.661874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:25.661874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 324/649] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:25.661874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0746c3-1480-4489-890b-9bc602bffab6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.678283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cb273-4ab2-40f6-ba28-9a732817e23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.678895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cb273-4ab2-40f6-ba28-9a732817e23b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.679179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cb273-4ab2-40f6-ba28-9a732817e23b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.679457 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cb273-4ab2-40f6-ba28-9a732817e23b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.680182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cb273-4ab2-40f6-ba28-9a732817e23b None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=218a015dd2b34435a76ecc8a763562ed) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:25.680484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cb273-4ab2-40f6-ba28-9a732817e23b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:25.681960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cb273-4ab2-40f6-ba28-9a732817e23b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:25.694967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 326/650] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/role_assignments?user.id=218a015dd2b34435a76ecc8a763562ed&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:25.694967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cb273-4ab2-40f6-ba28-9a732817e23b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.796206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0746c3-1480-4489-890b-9bc602bffab6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.805720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cb273-4ab2-40f6-ba28-9a732817e23b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:25.837336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4e2e6-523a-4c1a-b447-72731c8874bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.837961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4e2e6-523a-4c1a-b447-72731c8874bf None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.838243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4e2e6-523a-4c1a-b447-72731c8874bf None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.838518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4e2e6-523a-4c1a-b447-72731c8874bf None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.839257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4e2e6-523a-4c1a-b447-72731c8874bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:25.839569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4e2e6-523a-4c1a-b447-72731c8874bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:25.841115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4e2e6-523a-4c1a-b447-72731c8874bf None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:25.845357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad94a11-9beb-498b-90b2-bab3f6997292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:25.846016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad94a11-9beb-498b-90b2-bab3f6997292 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.846294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad94a11-9beb-498b-90b2-bab3f6997292 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.846563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad94a11-9beb-498b-90b2-bab3f6997292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.858702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad94a11-9beb-498b-90b2-bab3f6997292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:25.860024 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:25.860216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad94a11-9beb-498b-90b2-bab3f6997292 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:25.862227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:25.864644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 325/651] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:25.865526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4e2e6-523a-4c1a-b447-72731c8874bf None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6ad94a11-9beb-498b-90b2-bab3f6997292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:25.884762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:25.885799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:25.886749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:25.886749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:25.886749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:25.886749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 327/652] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:25.887442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f5d3b-f1dc-475f-bf84-7db5ae583e4a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:25.887543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f5d3b-f1dc-475f-bf84-7db5ae583e4a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:25.888405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f5d3b-f1dc-475f-bf84-7db5ae583e4a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:25.889265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 328/653] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:25.890003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f5d3b-f1dc-475f-bf84-7db5ae583e4a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:25.985815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4e2e6-523a-4c1a-b447-72731c8874bf None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.003794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f5d3b-f1dc-475f-bf84-7db5ae583e4a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.025223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbf40b-49f2-438b-8409-57c9fc79a3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.026285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbf40b-49f2-438b-8409-57c9fc79a3fc None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.026667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbf40b-49f2-438b-8409-57c9fc79a3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.026950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbf40b-49f2-438b-8409-57c9fc79a3fc None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.027714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbf40b-49f2-438b-8409-57c9fc79a3fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:26.028015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbf40b-49f2-438b-8409-57c9fc79a3fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.029536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbf40b-49f2-438b-8409-57c9fc79a3fc None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.039407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f1820-d918-4609-a981-9928bb4ea251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.040027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f1820-d918-4609-a981-9928bb4ea251 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.040309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f1820-d918-4609-a981-9928bb4ea251 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.040580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f1820-d918-4609-a981-9928bb4ea251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.044967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 326/654] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:26.044967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f86ce-da9e-439e-b80d-c8241fb3e549 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.044967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f86ce-da9e-439e-b80d-c8241fb3e549 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.044967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f86ce-da9e-439e-b80d-c8241fb3e549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.063606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-af8f86ce-da9e-439e-b80d-c8241fb3e549 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:26.064975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-af8f86ce-da9e-439e-b80d-c8241fb3e549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:26.115111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:26.124268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f1820-d918-4609-a981-9928bb4ea251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.125753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f1820-d918-4609-a981-9928bb4ea251 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.129616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:26.142887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 327/655] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:26.143553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f86ce-da9e-439e-b80d-c8241fb3e549 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.247506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f86ce-da9e-439e-b80d-c8241fb3e549 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.252671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 329/656] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:26.253764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f1820-d918-4609-a981-9928bb4ea251 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.290398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075328e-ea59-4182-91ed-b2db9e55c934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.291013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075328e-ea59-4182-91ed-b2db9e55c934 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.291295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075328e-ea59-4182-91ed-b2db9e55c934 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.291569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075328e-ea59-4182-91ed-b2db9e55c934 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.292219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075328e-ea59-4182-91ed-b2db9e55c934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:26.292526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075328e-ea59-4182-91ed-b2db9e55c934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.293995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075328e-ea59-4182-91ed-b2db9e55c934 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.306066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 328/657] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:26.306586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075328e-ea59-4182-91ed-b2db9e55c934 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.360419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:26.362192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:26.372449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f1820-d918-4609-a981-9928bb4ea251 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.410522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c69bd5-bba9-4071-bf18-4ccba18d1cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.411522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c69bd5-bba9-4071-bf18-4ccba18d1cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.412051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c69bd5-bba9-4071-bf18-4ccba18d1cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.412252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075328e-ea59-4182-91ed-b2db9e55c934 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.412733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c69bd5-bba9-4071-bf18-4ccba18d1cc3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.420269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c69bd5-bba9-4071-bf18-4ccba18d1cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.422709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c69bd5-bba9-4071-bf18-4ccba18d1cc3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-30c69bd5-bba9-4071-bf18-4ccba18d1cc3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:26.434770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:26.435765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:26.436764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 330/658] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:26.440054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c69bd5-bba9-4071-bf18-4ccba18d1cc3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.447864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57cc8a-5477-47d7-b4a1-c4f3d25b6909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.448502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57cc8a-5477-47d7-b4a1-c4f3d25b6909 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.448785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57cc8a-5477-47d7-b4a1-c4f3d25b6909 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.449090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57cc8a-5477-47d7-b4a1-c4f3d25b6909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.460252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57cc8a-5477-47d7-b4a1-c4f3d25b6909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.461744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57cc8a-5477-47d7-b4a1-c4f3d25b6909 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-0f57cc8a-5477-47d7-b4a1-c4f3d25b6909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:26.476773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:26.477813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:26.486201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:26.486201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:26.486201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:26.486201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 329/659] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:26.486201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57cc8a-5477-47d7-b4a1-c4f3d25b6909 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.556486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c69bd5-bba9-4071-bf18-4ccba18d1cc3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.591436 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a06bbe-c50e-4b9c-995b-bece02a682a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.592074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a06bbe-c50e-4b9c-995b-bece02a682a3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.592355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a06bbe-c50e-4b9c-995b-bece02a682a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.592671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a06bbe-c50e-4b9c-995b-bece02a682a3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.594776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57cc8a-5477-47d7-b4a1-c4f3d25b6909 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.596820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:26.601208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a06bbe-c50e-4b9c-995b-bece02a682a3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.602668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a06bbe-c50e-4b9c-995b-bece02a682a3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-11a06bbe-c50e-4b9c-995b-bece02a682a3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:26.615552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:26.616574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 10:43:26.617588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:26.621130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 331/660] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:26.621950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a06bbe-c50e-4b9c-995b-bece02a682a3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.633667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e170df7-f014-4eba-92e2-d9b4b6efaec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.634228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170df7-f014-4eba-92e2-d9b4b6efaec6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.634505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170df7-f014-4eba-92e2-d9b4b6efaec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.634773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170df7-f014-4eba-92e2-d9b4b6efaec6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.642372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:26.645770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e170df7-f014-4eba-92e2-d9b4b6efaec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-1e170df7-f014-4eba-92e2-d9b4b6efaec6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:26.650999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:26.652161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:26.653249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:26.656965 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 330/661] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:26.657710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e170df7-f014-4eba-92e2-d9b4b6efaec6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.745870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a06bbe-c50e-4b9c-995b-bece02a682a3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.770790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e170df7-f014-4eba-92e2-d9b4b6efaec6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.787542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360403f-2ba6-49a2-8b88-19017b9e7dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.788426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360403f-2ba6-49a2-8b88-19017b9e7dfe None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.788897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360403f-2ba6-49a2-8b88-19017b9e7dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.789417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360403f-2ba6-49a2-8b88-19017b9e7dfe None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.790714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360403f-2ba6-49a2-8b88-19017b9e7dfe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:26.791445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360403f-2ba6-49a2-8b88-19017b9e7dfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.793536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360403f-2ba6-49a2-8b88-19017b9e7dfe None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.804067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd09d1-8be9-49b5-93c8-ceae77168c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.804670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd09d1-8be9-49b5-93c8-ceae77168c19 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.804980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd09d1-8be9-49b5-93c8-ceae77168c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.805255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd09d1-8be9-49b5-93c8-ceae77168c19 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.805888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd09d1-8be9-49b5-93c8-ceae77168c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:26.806193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd09d1-8be9-49b5-93c8-ceae77168c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.807549 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd09d1-8be9-49b5-93c8-ceae77168c19 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.817542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 332/662] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:26.818856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360403f-2ba6-49a2-8b88-19017b9e7dfe None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.823963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 331/663] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:26.823963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd09d1-8be9-49b5-93c8-ceae77168c19 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.848713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:26.894731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:26.929434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360403f-2ba6-49a2-8b88-19017b9e7dfe None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.932010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd09d1-8be9-49b5-93c8-ceae77168c19 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:26.967094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc112ea-9bc1-45c9-b197-ae228eb6e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.967821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc112ea-9bc1-45c9-b197-ae228eb6e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.968169 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc112ea-9bc1-45c9-b197-ae228eb6e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.968277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a2768-de95-4c34-aa09-0038a3b7617a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:26.968559 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc112ea-9bc1-45c9-b197-ae228eb6e9f2 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.968790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a2768-de95-4c34-aa09-0038a3b7617a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:26.969096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a2768-de95-4c34-aa09-0038a3b7617a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:26.969367 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a2768-de95-4c34-aa09-0038a3b7617a None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:26.969573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc112ea-9bc1-45c9-b197-ae228eb6e9f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:26.969945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc112ea-9bc1-45c9-b197-ae228eb6e9f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.970043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a2768-de95-4c34-aa09-0038a3b7617a None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:26.970310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a2768-de95-4c34-aa09-0038a3b7617a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:26.971597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc112ea-9bc1-45c9-b197-ae228eb6e9f2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.971700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a2768-de95-4c34-aa09-0038a3b7617a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:26.980028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 332/664] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:26.985261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a2768-de95-4c34-aa09-0038a3b7617a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:26.990621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 333/665] 10.222.0.50 () {56 vars in 1139 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:26.991638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc112ea-9bc1-45c9-b197-ae228eb6e9f2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:27.080851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:27.098610 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a2768-de95-4c34-aa09-0038a3b7617a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:27.102332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc112ea-9bc1-45c9-b197-ae228eb6e9f2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:27.133268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:27.134841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dde5c-a8e9-456c-a16d-c0f11f1ae188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:27.135567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dde5c-a8e9-456c-a16d-c0f11f1ae188 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:27.136006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dde5c-a8e9-456c-a16d-c0f11f1ae188 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:27.136392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dde5c-a8e9-456c-a16d-c0f11f1ae188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:27.139934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b659e-b2ab-4a4b-aae5-d460973741a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:27.140574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b659e-b2ab-4a4b-aae5-d460973741a7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:27.140884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b659e-b2ab-4a4b-aae5-d460973741a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:27.141168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b659e-b2ab-4a4b-aae5-d460973741a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:27.147915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b659e-b2ab-4a4b-aae5-d460973741a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:27.149403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b659e-b2ab-4a4b-aae5-d460973741a7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6d2b659e-b2ab-4a4b-aae5-d460973741a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:27.161350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:27.162239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:27.163137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:27.163137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:27.163137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:27.163137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:27.163137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:27.163137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:27.163137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:27.163137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:27.163137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 334/666] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:27.163775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b659e-b2ab-4a4b-aae5-d460973741a7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:27.215896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dde5c-a8e9-456c-a16d-c0f11f1ae188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:27.218116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dde5c-a8e9-456c-a16d-c0f11f1ae188 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:27.272676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b659e-b2ab-4a4b-aae5-d460973741a7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:27.305291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:27.306255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c1bc9-f39d-4c58-ae0b-1f2e8846b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:27.306912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c1bc9-f39d-4c58-ae0b-1f2e8846b258 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:27.307272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c1bc9-f39d-4c58-ae0b-1f2e8846b258 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:27.307609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c1bc9-f39d-4c58-ae0b-1f2e8846b258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:27.318169 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c1bc9-f39d-4c58-ae0b-1f2e8846b258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:27.319617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c1bc9-f39d-4c58-ae0b-1f2e8846b258 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:27.330126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 333/667] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:27.330126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dde5c-a8e9-456c-a16d-c0f11f1ae188 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ac7c1bc9-f39d-4c58-ae0b-1f2e8846b258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:27.339496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:27.340497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:27.341510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:27.341510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:27.341510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:27.341510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 335/668] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:27.343338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c1bc9-f39d-4c58-ae0b-1f2e8846b258 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:27.386727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:27.440111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dde5c-a8e9-456c-a16d-c0f11f1ae188 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:27.445215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c1bc9-f39d-4c58-ae0b-1f2e8846b258 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:27.476397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d38bf0-67e9-4abe-b110-6bb24710598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:27.477136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d38bf0-67e9-4abe-b110-6bb24710598d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:27.477427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d38bf0-67e9-4abe-b110-6bb24710598d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:27.477688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d38bf0-67e9-4abe-b110-6bb24710598d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:27.481419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0c25d-96d4-4bbb-bdae-b29ee596a7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:27.482114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0c25d-96d4-4bbb-bdae-b29ee596a7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:27.482396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0c25d-96d4-4bbb-bdae-b29ee596a7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:27.482666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0c25d-96d4-4bbb-bdae-b29ee596a7e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:27.483974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0c25d-96d4-4bbb-bdae-b29ee596a7e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:27.484357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0c25d-96d4-4bbb-bdae-b29ee596a7e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:27.485760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0c25d-96d4-4bbb-bdae-b29ee596a7e6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:27.498774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 336/669] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:27.500476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0c25d-96d4-4bbb-bdae-b29ee596a7e6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:27.543917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:27.564770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d38bf0-67e9-4abe-b110-6bb24710598d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:27.566282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d38bf0-67e9-4abe-b110-6bb24710598d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:27.610141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0c25d-96d4-4bbb-bdae-b29ee596a7e6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:27.611864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:27.649042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95073f07-ecd5-45d6-9c47-3a38b2d02628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:27.649680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95073f07-ecd5-45d6-9c47-3a38b2d02628 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:27.649964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95073f07-ecd5-45d6-9c47-3a38b2d02628 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:27.650239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95073f07-ecd5-45d6-9c47-3a38b2d02628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:27.661129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95073f07-ecd5-45d6-9c47-3a38b2d02628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:27.662602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95073f07-ecd5-45d6-9c47-3a38b2d02628 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:27.677189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 334/670] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:27.677189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d38bf0-67e9-4abe-b110-6bb24710598d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-95073f07-ecd5-45d6-9c47-3a38b2d02628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:27.677614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:27.678582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:27.679492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:27.679492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:27.679492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:27.679492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 337/671] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:27.679732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95073f07-ecd5-45d6-9c47-3a38b2d02628 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:27.775123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:27.788322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95073f07-ecd5-45d6-9c47-3a38b2d02628 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:27.794416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d38bf0-67e9-4abe-b110-6bb24710598d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:27.825720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e9645-7cc9-4bf2-923a-5f767c92f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:27.826327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e9645-7cc9-4bf2-923a-5f767c92f22d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:27.826620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e9645-7cc9-4bf2-923a-5f767c92f22d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:27.827730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e9645-7cc9-4bf2-923a-5f767c92f22d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:27.828460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e9645-7cc9-4bf2-923a-5f767c92f22d None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=6908a326a3d3437c9db898976a2def1b) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:27.828773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e9645-7cc9-4bf2-923a-5f767c92f22d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:27.830255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e9645-7cc9-4bf2-923a-5f767c92f22d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:27.833569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469cda4-58cb-46eb-b60e-37931da070a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:27.834173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469cda4-58cb-46eb-b60e-37931da070a5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:27.834452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469cda4-58cb-46eb-b60e-37931da070a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:27.834753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469cda4-58cb-46eb-b60e-37931da070a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:27.835390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469cda4-58cb-46eb-b60e-37931da070a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:27.835690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469cda4-58cb-46eb-b60e-37931da070a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:27.836632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 338/672] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:27 2022] GET /identity/v3/role_assignments?user.id=6908a326a3d3437c9db898976a2def1b&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:27.837199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469cda4-58cb-46eb-b60e-37931da070a5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:27.841784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e9645-7cc9-4bf2-923a-5f767c92f22d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:27.854503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 335/673] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:27.855642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469cda4-58cb-46eb-b60e-37931da070a5 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:27.876969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:27.988087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469cda4-58cb-46eb-b60e-37931da070a5 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:27.992686 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e9645-7cc9-4bf2-923a-5f767c92f22d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.033544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4155ffa-0e2c-4419-a539-3b0ceb82b406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.033734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ad3e5-a255-4ad7-a35d-1c489f831965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.034086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4155ffa-0e2c-4419-a539-3b0ceb82b406 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.034211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad3e5-a255-4ad7-a35d-1c489f831965 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.034372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4155ffa-0e2c-4419-a539-3b0ceb82b406 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.034523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad3e5-a255-4ad7-a35d-1c489f831965 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.034701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4155ffa-0e2c-4419-a539-3b0ceb82b406 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.034801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad3e5-a255-4ad7-a35d-1c489f831965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.035292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4155ffa-0e2c-4419-a539-3b0ceb82b406 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:28.035603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4155ffa-0e2c-4419-a539-3b0ceb82b406 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:28.037025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4155ffa-0e2c-4419-a539-3b0ceb82b406 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:28.047773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ad3e5-a255-4ad7-a35d-1c489f831965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:28.049314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ad3e5-a255-4ad7-a35d-1c489f831965 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:28.050908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 336/674] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:28.051813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4155ffa-0e2c-4419-a539-3b0ceb82b406 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.057653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ef5ad3e5-a255-4ad7-a35d-1c489f831965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:28.067994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:28.069028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:28.071342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:28.071342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:28.071342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:28.071342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 339/675] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:28.071575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6a512-8ec7-4c00-bc8b-4669a979717a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.071806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6a512-8ec7-4c00-bc8b-4669a979717a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.072080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6a512-8ec7-4c00-bc8b-4669a979717a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.073508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 340/676] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:28.074065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6a512-8ec7-4c00-bc8b-4669a979717a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.154633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:28.177025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4155ffa-0e2c-4419-a539-3b0ceb82b406 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.188619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6a512-8ec7-4c00-bc8b-4669a979717a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.219554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d924cb7-3b72-4d91-b648-c1a11262b0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.221213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d924cb7-3b72-4d91-b648-c1a11262b0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.221679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d924cb7-3b72-4d91-b648-c1a11262b0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.222156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d924cb7-3b72-4d91-b648-c1a11262b0a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.224942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bbee0-e49e-4153-80e7-c0d856b58ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.225571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bbee0-e49e-4153-80e7-c0d856b58ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.225853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bbee0-e49e-4153-80e7-c0d856b58ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.226126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bbee0-e49e-4153-80e7-c0d856b58ab4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.230641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d924cb7-3b72-4d91-b648-c1a11262b0a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:28.232284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d924cb7-3b72-4d91-b648-c1a11262b0a5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:28.232857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bbee0-e49e-4153-80e7-c0d856b58ab4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:28.234581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bbee0-e49e-4153-80e7-c0d856b58ab4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-8d924cb7-3b72-4d91-b648-c1a11262b0a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:28.245981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:28.246961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:28.247937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3b1bbee0-e49e-4153-80e7-c0d856b58ab4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:28.248494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:28.249628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 341/677] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:28.255839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bbee0-e49e-4153-80e7-c0d856b58ab4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.256428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 337/678] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:28.256428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d296320-b687-457c-b8a0-63381c97181c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.256428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d296320-b687-457c-b8a0-63381c97181c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.256428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d296320-b687-457c-b8a0-63381c97181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.273120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-1d296320-b687-457c-b8a0-63381c97181c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:28.274866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-1d296320-b687-457c-b8a0-63381c97181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:28.342064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:28.356996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 338/679] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:28.356996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d296320-b687-457c-b8a0-63381c97181c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.367346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bbee0-e49e-4153-80e7-c0d856b58ab4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.411042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:28.419246 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d193a-dd2e-4307-8e0c-ae1ee0daf9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.420112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d193a-dd2e-4307-8e0c-ae1ee0daf9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.420396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d193a-dd2e-4307-8e0c-ae1ee0daf9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.421406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d193a-dd2e-4307-8e0c-ae1ee0daf9c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.422641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d193a-dd2e-4307-8e0c-ae1ee0daf9c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:28.422959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d193a-dd2e-4307-8e0c-ae1ee0daf9c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:28.424347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d193a-dd2e-4307-8e0c-ae1ee0daf9c9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:28.445312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 342/680] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:28.446196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d193a-dd2e-4307-8e0c-ae1ee0daf9c9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.487003 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d296320-b687-457c-b8a0-63381c97181c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.527622 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b2053-f1bc-455f-a97a-d64db01a1a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.528719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b2053-f1bc-455f-a97a-d64db01a1a10 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.529339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b2053-f1bc-455f-a97a-d64db01a1a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.529764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b2053-f1bc-455f-a97a-d64db01a1a10 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.531437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b2053-f1bc-455f-a97a-d64db01a1a10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:28.531898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b2053-f1bc-455f-a97a-d64db01a1a10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:28.533919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b2053-f1bc-455f-a97a-d64db01a1a10 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:28.554246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 339/681] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:28.555585 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b2053-f1bc-455f-a97a-d64db01a1a10 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.596405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d193a-dd2e-4307-8e0c-ae1ee0daf9c9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.629602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:28.638499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a33ad-5a31-4900-bcad-cf3a395e5e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.639425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a33ad-5a31-4900-bcad-cf3a395e5e6a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.639896 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a33ad-5a31-4900-bcad-cf3a395e5e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.640352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a33ad-5a31-4900-bcad-cf3a395e5e6a None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.641298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a33ad-5a31-4900-bcad-cf3a395e5e6a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:28.641791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a33ad-5a31-4900-bcad-cf3a395e5e6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:28.643436 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a33ad-5a31-4900-bcad-cf3a395e5e6a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:28.663196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 343/682] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:28.664442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a33ad-5a31-4900-bcad-cf3a395e5e6a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.679860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b2053-f1bc-455f-a97a-d64db01a1a10 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.719909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:28.722482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156fb7-f3e1-4905-a21e-79cb3ff8f0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.723349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156fb7-f3e1-4905-a21e-79cb3ff8f0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.723814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156fb7-f3e1-4905-a21e-79cb3ff8f0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.724282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156fb7-f3e1-4905-a21e-79cb3ff8f0a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.732424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51156fb7-f3e1-4905-a21e-79cb3ff8f0a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:28.734502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51156fb7-f3e1-4905-a21e-79cb3ff8f0a1 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-51156fb7-f3e1-4905-a21e-79cb3ff8f0a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:28.746174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:28.747091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 340/683] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:28.748002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156fb7-f3e1-4905-a21e-79cb3ff8f0a1 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.781302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a33ad-5a31-4900-bcad-cf3a395e5e6a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.821715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5359a2-b188-40c9-a7bb-a956955024db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.822332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5359a2-b188-40c9-a7bb-a956955024db None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.822617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5359a2-b188-40c9-a7bb-a956955024db None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.822892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5359a2-b188-40c9-a7bb-a956955024db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.823881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5359a2-b188-40c9-a7bb-a956955024db None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=d7fa5d550a2d442a820cf4e12ebc36c5) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:28.824192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5359a2-b188-40c9-a7bb-a956955024db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:28.826573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5359a2-b188-40c9-a7bb-a956955024db None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:28.845149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 344/684] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/role_assignments?user.id=d7fa5d550a2d442a820cf4e12ebc36c5&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:28.845149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5359a2-b188-40c9-a7bb-a956955024db None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.862869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156fb7-f3e1-4905-a21e-79cb3ff8f0a1 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.879582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:28.904328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb10edd-137e-4a05-b578-ac635e73ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:28.905628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb10edd-137e-4a05-b578-ac635e73ef6e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:28.905914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb10edd-137e-4a05-b578-ac635e73ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:28.906184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb10edd-137e-4a05-b578-ac635e73ef6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:28.919308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb10edd-137e-4a05-b578-ac635e73ef6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-efb10edd-137e-4a05-b578-ac635e73ef6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:28.926158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:28.927342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:28.928194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 341/685] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:28.928943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb10edd-137e-4a05-b578-ac635e73ef6e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:28.972930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5359a2-b188-40c9-a7bb-a956955024db None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:28.988048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:29.013975 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6489f6-398c-46e2-b7e8-16366fff7d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:29.015300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6489f6-398c-46e2-b7e8-16366fff7d78 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:29.015646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6489f6-398c-46e2-b7e8-16366fff7d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:29.015921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6489f6-398c-46e2-b7e8-16366fff7d78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:29.050168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb10edd-137e-4a05-b578-ac635e73ef6e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:29.096447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61488d-f155-43d3-8366-9aa92d97e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:29.097219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61488d-f155-43d3-8366-9aa92d97e822 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:29.097543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61488d-f155-43d3-8366-9aa92d97e822 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:29.097855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61488d-f155-43d3-8366-9aa92d97e822 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:29.098587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61488d-f155-43d3-8366-9aa92d97e822 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:29.099048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61488d-f155-43d3-8366-9aa92d97e822 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:29.102478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61488d-f155-43d3-8366-9aa92d97e822 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:29.108364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6489f6-398c-46e2-b7e8-16366fff7d78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:29.109842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6489f6-398c-46e2-b7e8-16366fff7d78 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:29.117626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 342/686] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:29.118409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61488d-f155-43d3-8366-9aa92d97e822 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:29.161158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:29.252662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 345/687] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:29.253596 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6489f6-398c-46e2-b7e8-16366fff7d78 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:29.275903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:29.288678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61488d-f155-43d3-8366-9aa92d97e822 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:29.344585 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe41b2f-88fd-4325-9841-5fd27c2709f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:29.345587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe41b2f-88fd-4325-9841-5fd27c2709f4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:29.345905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe41b2f-88fd-4325-9841-5fd27c2709f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:29.346502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe41b2f-88fd-4325-9841-5fd27c2709f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:29.347020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe41b2f-88fd-4325-9841-5fd27c2709f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:29.347370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe41b2f-88fd-4325-9841-5fd27c2709f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:29.348779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe41b2f-88fd-4325-9841-5fd27c2709f4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:29.361685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 343/688] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:29.362646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe41b2f-88fd-4325-9841-5fd27c2709f4 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:29.397294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6489f6-398c-46e2-b7e8-16366fff7d78 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:29.449366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a8e4-6ec8-478f-af0b-69c056c5fcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:29.450380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a8e4-6ec8-478f-af0b-69c056c5fcaa None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:29.451068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a8e4-6ec8-478f-af0b-69c056c5fcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:29.451502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a8e4-6ec8-478f-af0b-69c056c5fcaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:29.453531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a8e4-6ec8-478f-af0b-69c056c5fcaa None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=f5e7b08dc3544e76ac6732ca2aa89bf5) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:29.453838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a8e4-6ec8-478f-af0b-69c056c5fcaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:29.455367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a8e4-6ec8-478f-af0b-69c056c5fcaa None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:29.466437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 346/689] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:29 2022] GET /identity/v3/role_assignments?user.id=f5e7b08dc3544e76ac6732ca2aa89bf5&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:29.467943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a8e4-6ec8-478f-af0b-69c056c5fcaa None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:29.505486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:29.516183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe41b2f-88fd-4325-9841-5fd27c2709f4 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:29.573302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738331b-e20b-4449-9199-61ec908234a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:29.573950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738331b-e20b-4449-9199-61ec908234a9 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:29.574227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738331b-e20b-4449-9199-61ec908234a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:29.574496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738331b-e20b-4449-9199-61ec908234a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:29.605313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:29.605951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738331b-e20b-4449-9199-61ec908234a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:29.607447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738331b-e20b-4449-9199-61ec908234a9 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:29.634562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a8e4-6ec8-478f-af0b-69c056c5fcaa None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-2738331b-e20b-4449-9199-61ec908234a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:29.653185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:29.654189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:29.656389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:29.656389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:29.656389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:29.656389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 344/690] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:29.661695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738331b-e20b-4449-9199-61ec908234a9 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:29.700019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304d7d-0205-42b6-8e4c-166b818007d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:29.700697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304d7d-0205-42b6-8e4c-166b818007d4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:29.701003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304d7d-0205-42b6-8e4c-166b818007d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:29.701272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304d7d-0205-42b6-8e4c-166b818007d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:29.733570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82304d7d-0205-42b6-8e4c-166b818007d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:29.735128 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82304d7d-0205-42b6-8e4c-166b818007d4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-82304d7d-0205-42b6-8e4c-166b818007d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:29.764928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:29.765955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:29.766955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:29.766955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:29.766955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:29.769212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 347/691] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:29.777437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304d7d-0205-42b6-8e4c-166b818007d4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:29.824869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738331b-e20b-4449-9199-61ec908234a9 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:29.870391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3359b-6729-456f-8181-4fe13096f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:29.871968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3359b-6729-456f-8181-4fe13096f319 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:29.872254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3359b-6729-456f-8181-4fe13096f319 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:29.872529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3359b-6729-456f-8181-4fe13096f319 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:29.873226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3359b-6729-456f-8181-4fe13096f319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:29.873562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3359b-6729-456f-8181-4fe13096f319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:29.875694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3359b-6729-456f-8181-4fe13096f319 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:29.892535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 345/692] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:29.898004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3359b-6729-456f-8181-4fe13096f319 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:29.913073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:29.922759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304d7d-0205-42b6-8e4c-166b818007d4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:29.958740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:29.973844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc95a-1e11-449e-8c4c-20b99f6e063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:29.974488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc95a-1e11-449e-8c4c-20b99f6e063a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:29.974775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc95a-1e11-449e-8c4c-20b99f6e063a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:29.975054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc95a-1e11-449e-8c4c-20b99f6e063a None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:29.975789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc95a-1e11-449e-8c4c-20b99f6e063a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:29.976103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc95a-1e11-449e-8c4c-20b99f6e063a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:29.979723 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc95a-1e11-449e-8c4c-20b99f6e063a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:30.004250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 348/693] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:30.010392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc95a-1e11-449e-8c4c-20b99f6e063a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:30.063996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3359b-6729-456f-8181-4fe13096f319 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:30.126899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65e238-f9ce-4825-8615-4c88fc828001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:30.127538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65e238-f9ce-4825-8615-4c88fc828001 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:30.127821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65e238-f9ce-4825-8615-4c88fc828001 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:30.128097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65e238-f9ce-4825-8615-4c88fc828001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:30.140513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65e238-f9ce-4825-8615-4c88fc828001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:30.142071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65e238-f9ce-4825-8615-4c88fc828001 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-2b65e238-f9ce-4825-8615-4c88fc828001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:30.161037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:30.163256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:30.165452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:30.165452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:30.165452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:30.165452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:30.165452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:30.165452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:30.165452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:30.165452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:30.166612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 346/694] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:30.168834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65e238-f9ce-4825-8615-4c88fc828001 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:30.174385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc95a-1e11-449e-8c4c-20b99f6e063a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:30.255594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277ab4a-18a9-447c-8ffb-97244a023b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:30.256375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277ab4a-18a9-447c-8ffb-97244a023b48 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:30.257561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277ab4a-18a9-447c-8ffb-97244a023b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:30.257841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277ab4a-18a9-447c-8ffb-97244a023b48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:30.271434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277ab4a-18a9-447c-8ffb-97244a023b48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:30.273627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277ab4a-18a9-447c-8ffb-97244a023b48 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:30.295241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-1277ab4a-18a9-447c-8ffb-97244a023b48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:30.300547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:30.302769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:30.305151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:30.307172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 349/695] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Sep 15 10:43:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:30.314497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277ab4a-18a9-447c-8ffb-97244a023b48 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:30.354719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65e238-f9ce-4825-8615-4c88fc828001 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:30.389210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:30.390554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7a6a6-ce27-4f9f-8b98-c91054c7d005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:30.391170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7a6a6-ce27-4f9f-8b98-c91054c7d005 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:30.391449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7a6a6-ce27-4f9f-8b98-c91054c7d005 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:30.391720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7a6a6-ce27-4f9f-8b98-c91054c7d005 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:30.392413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7a6a6-ce27-4f9f-8b98-c91054c7d005 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:30.392829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7a6a6-ce27-4f9f-8b98-c91054c7d005 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:30.394417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7a6a6-ce27-4f9f-8b98-c91054c7d005 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:30.407713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 347/696] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:30.413777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7726a-cf30-4220-a0c7-0335bd0ee4eb None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:30.414025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7726a-cf30-4220-a0c7-0335bd0ee4eb None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:30.414303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7726a-cf30-4220-a0c7-0335bd0ee4eb None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:30.415826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 348/697] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:30.416549 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7726a-cf30-4220-a0c7-0335bd0ee4eb None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:30.434265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277ab4a-18a9-447c-8ffb-97244a023b48 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:30.484451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c89fe4-9a7c-4971-8f31-18553125841f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:30.485087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c89fe4-9a7c-4971-8f31-18553125841f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:30.485369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c89fe4-9a7c-4971-8f31-18553125841f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:30.485801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c89fe4-9a7c-4971-8f31-18553125841f None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:30.486848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c89fe4-9a7c-4971-8f31-18553125841f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:30.487161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c89fe4-9a7c-4971-8f31-18553125841f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:30.489036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c89fe4-9a7c-4971-8f31-18553125841f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:30.521075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 350/698] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Sep 15 10:43:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:30.522404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bba5a-9709-4a7e-93f5-4142ea7c8f7d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:30.522689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bba5a-9709-4a7e-93f5-4142ea7c8f7d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:30.522960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bba5a-9709-4a7e-93f5-4142ea7c8f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:30.543435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0e3bba5a-9709-4a7e-93f5-4142ea7c8f7d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:30.546160 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0e3bba5a-9709-4a7e-93f5-4142ea7c8f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:30.548574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7726a-cf30-4220-a0c7-0335bd0ee4eb None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:30.581464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:30.590535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477faf84-4f11-42aa-ba0c-29bb02ef1068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:30.591200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477faf84-4f11-42aa-ba0c-29bb02ef1068 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:30.591563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477faf84-4f11-42aa-ba0c-29bb02ef1068 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:30.591896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477faf84-4f11-42aa-ba0c-29bb02ef1068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:30.592880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477faf84-4f11-42aa-ba0c-29bb02ef1068 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=06007b3b2260409f863ac3d0ac85f4cc) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:30.593625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477faf84-4f11-42aa-ba0c-29bb02ef1068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:30.594944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477faf84-4f11-42aa-ba0c-29bb02ef1068 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:30.604956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 349/699] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:30 2022] GET /identity/v3/role_assignments?user.id=06007b3b2260409f863ac3d0ac85f4cc&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:30.606121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477faf84-4f11-42aa-ba0c-29bb02ef1068 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:30.646951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 351/700] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:30.752296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477faf84-4f11-42aa-ba0c-29bb02ef1068 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:30.798856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cea60-43b6-4e09-b7a8-13bff6e4f1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:30.799833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cea60-43b6-4e09-b7a8-13bff6e4f1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:30.800329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cea60-43b6-4e09-b7a8-13bff6e4f1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:30.800826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cea60-43b6-4e09-b7a8-13bff6e4f1d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:30.879568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:30.889914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cea60-43b6-4e09-b7a8-13bff6e4f1d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:30.891393 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cea60-43b6-4e09-b7a8-13bff6e4f1d2 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:30.994841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 350/701] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:31.001647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bba5a-9709-4a7e-93f5-4142ea7c8f7d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:31.039136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:31.104678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cea60-43b6-4e09-b7a8-13bff6e4f1d2 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:31.112949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bba5a-9709-4a7e-93f5-4142ea7c8f7d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:31.158368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc70cf5-ff2b-40d6-a92d-36f717da1888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:31.159389 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc70cf5-ff2b-40d6-a92d-36f717da1888 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:31.159886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc70cf5-ff2b-40d6-a92d-36f717da1888 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:31.160361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc70cf5-ff2b-40d6-a92d-36f717da1888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:31.171460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc70cf5-ff2b-40d6-a92d-36f717da1888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:31.173360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc70cf5-ff2b-40d6-a92d-36f717da1888 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5bc70cf5-ff2b-40d6-a92d-36f717da1888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:31.190518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:31.192311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:31.194267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:31.194267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:31.194267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:31.197843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 352/702] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:31.204316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc70cf5-ff2b-40d6-a92d-36f717da1888 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:31.218581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:31.234475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cea60-43b6-4e09-b7a8-13bff6e4f1d2 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:31.276374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c16a9-5c82-4c0f-89b3-f4293fe9e9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:31.277081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c16a9-5c82-4c0f-89b3-f4293fe9e9ab None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:31.277366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c16a9-5c82-4c0f-89b3-f4293fe9e9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:31.277644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c16a9-5c82-4c0f-89b3-f4293fe9e9ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:31.291557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c16a9-5c82-4c0f-89b3-f4293fe9e9ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:31.293016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c16a9-5c82-4c0f-89b3-f4293fe9e9ab None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-5f5c16a9-5c82-4c0f-89b3-f4293fe9e9ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:31.314291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:31.315767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:31.316692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:31.316692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:31.316692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:31.320635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 351/703] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:31.322072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c16a9-5c82-4c0f-89b3-f4293fe9e9ab None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:31.333944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:31.352715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc70cf5-ff2b-40d6-a92d-36f717da1888 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:31.399555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb24ae-d21d-48cd-8c4d-6ea42e264c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:31.400748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb24ae-d21d-48cd-8c4d-6ea42e264c05 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:31.401156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb24ae-d21d-48cd-8c4d-6ea42e264c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:31.401649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb24ae-d21d-48cd-8c4d-6ea42e264c05 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:31.402543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb24ae-d21d-48cd-8c4d-6ea42e264c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:31.402950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb24ae-d21d-48cd-8c4d-6ea42e264c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:31.404477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb24ae-d21d-48cd-8c4d-6ea42e264c05 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:31.420906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 353/704] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:31.429777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb24ae-d21d-48cd-8c4d-6ea42e264c05 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:31.457096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c16a9-5c82-4c0f-89b3-f4293fe9e9ab None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:31.495111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e94805-ee25-42af-9f79-2c5f202b7a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:31.496048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e94805-ee25-42af-9f79-2c5f202b7a34 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:31.496546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e94805-ee25-42af-9f79-2c5f202b7a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:31.497001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e94805-ee25-42af-9f79-2c5f202b7a34 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:31.497789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e94805-ee25-42af-9f79-2c5f202b7a34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:31.498279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e94805-ee25-42af-9f79-2c5f202b7a34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:31.500646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e94805-ee25-42af-9f79-2c5f202b7a34 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:31.518561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 352/705] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:31.520223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393151c-9842-4879-b577-7d0b01429e32 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:31.520653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393151c-9842-4879-b577-7d0b01429e32 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:31.521138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393151c-9842-4879-b577-7d0b01429e32 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:31.523296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 353/706] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:31.524252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393151c-9842-4879-b577-7d0b01429e32 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:31.531184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:31.568472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb24ae-d21d-48cd-8c4d-6ea42e264c05 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:31.618325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8aaf5-8ead-454c-a48a-659461c2d47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:31.619228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8aaf5-8ead-454c-a48a-659461c2d47e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:31.619516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8aaf5-8ead-454c-a48a-659461c2d47e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:31.619789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8aaf5-8ead-454c-a48a-659461c2d47e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:31.629955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8aaf5-8ead-454c-a48a-659461c2d47e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:31.631586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8aaf5-8ead-454c-a48a-659461c2d47e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-22e8aaf5-8ead-454c-a48a-659461c2d47e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:31.647578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:31.648530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:31.649802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:31.649802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:31.649802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:31.649802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:31.649802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:31.649802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:31.649802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:31.649802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:31.649802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 354/707] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:31.650578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9e464-ef78-4194-935d-5b13955daaad None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:31.650674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9e464-ef78-4194-935d-5b13955daaad None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:31.650910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9e464-ef78-4194-935d-5b13955daaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:31.666599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393151c-9842-4879-b577-7d0b01429e32 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:31.676312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-57e9e464-ef78-4194-935d-5b13955daaad None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:31.677852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-57e9e464-ef78-4194-935d-5b13955daaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:31.680667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:31.711927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e32561-23f3-4abb-8646-a8bfe778ba97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:31.712974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32561-23f3-4abb-8646-a8bfe778ba97 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:31.713486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32561-23f3-4abb-8646-a8bfe778ba97 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:31.713963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32561-23f3-4abb-8646-a8bfe778ba97 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:31.714810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e32561-23f3-4abb-8646-a8bfe778ba97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:31.716628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e32561-23f3-4abb-8646-a8bfe778ba97 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:31.749270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 354/708] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:43:31 2022] POST /identity/v3/users => generated 254 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:31.750966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e32561-23f3-4abb-8646-a8bfe778ba97 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:31.777768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 355/709] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:31.838785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:31.891755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e32561-23f3-4abb-8646-a8bfe778ba97 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:31.931407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1631-2887-4ab3-a01d-b9ea8e291c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:31.932033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1631-2887-4ab3-a01d-b9ea8e291c33 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:31.932316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1631-2887-4ab3-a01d-b9ea8e291c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:31.932592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1631-2887-4ab3-a01d-b9ea8e291c33 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:31.933336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1631-2887-4ab3-a01d-b9ea8e291c33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:31.933650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1631-2887-4ab3-a01d-b9ea8e291c33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:31.935110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1631-2887-4ab3-a01d-b9ea8e291c33 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:31.950183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 355/710] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:31.957152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9e464-ef78-4194-935d-5b13955daaad None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:32.097006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9e464-ef78-4194-935d-5b13955daaad None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:32.142112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b302dd-7600-4cb5-b2c8-1a5f2467ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:32.142774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b302dd-7600-4cb5-b2c8-1a5f2467ad98 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:32.143062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b302dd-7600-4cb5-b2c8-1a5f2467ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:32.143336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b302dd-7600-4cb5-b2c8-1a5f2467ad98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:32.144061 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b302dd-7600-4cb5-b2c8-1a5f2467ad98 None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=218a015dd2b34435a76ecc8a763562ed) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:32.144370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b302dd-7600-4cb5-b2c8-1a5f2467ad98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:32.145840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b302dd-7600-4cb5-b2c8-1a5f2467ad98 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:32.152168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 356/711] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:31 2022] GET /identity/v3/role_assignments?user.id=218a015dd2b34435a76ecc8a763562ed&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=0be9777002c641b4a47f78df647cc173 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:32.207499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1631-2887-4ab3-a01d-b9ea8e291c33 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:32.235266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059e61c-aefc-46fb-9d62-90a802fa28cb None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:32.235266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059e61c-aefc-46fb-9d62-90a802fa28cb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:32.235266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059e61c-aefc-46fb-9d62-90a802fa28cb None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:32.236978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 357/712] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:43:32.242794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082a91-75ae-415b-b2a9-6943301d6f67 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:32.243094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082a91-75ae-415b-b2a9-6943301d6f67 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:32.243354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082a91-75ae-415b-b2a9-6943301d6f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:32.256575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:32.275741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-36082a91-75ae-415b-b2a9-6943301d6f67 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:32.278152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-36082a91-75ae-415b-b2a9-6943301d6f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:32.369802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1631-2887-4ab3-a01d-b9ea8e291c33 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:32.382112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 358/713] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:32.401719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:32.428265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ed12-5f64-4612-817c-b3d9c1eda329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:32.429321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ed12-5f64-4612-817c-b3d9c1eda329 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:32.429822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ed12-5f64-4612-817c-b3d9c1eda329 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:32.430310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ed12-5f64-4612-817c-b3d9c1eda329 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:32.431106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563ed12-5f64-4612-817c-b3d9c1eda329 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:32.432696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563ed12-5f64-4612-817c-b3d9c1eda329 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-7563ed12-5f64-4612-817c-b3d9c1eda329 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:32.445154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:32.446228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 10:43:32.447191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:32.449970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 356/714] 10.222.0.50 () {56 vars in 1129 bytes} [Thu Sep 15 10:43:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:32.456620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082a91-75ae-415b-b2a9-6943301d6f67 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:32.592582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082a91-75ae-415b-b2a9-6943301d6f67 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:32.638119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:32.643465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9f89c-55fb-4bd4-b0a8-66d5bbb63bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:32.644586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f89c-55fb-4bd4-b0a8-66d5bbb63bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:32.644926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f89c-55fb-4bd4-b0a8-66d5bbb63bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:32.645627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f89c-55fb-4bd4-b0a8-66d5bbb63bf3 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:32.646523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9f89c-55fb-4bd4-b0a8-66d5bbb63bf3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:32.647225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9f89c-55fb-4bd4-b0a8-66d5bbb63bf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:32.648621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9f89c-55fb-4bd4-b0a8-66d5bbb63bf3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:32.654197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 359/715] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Sep 15 10:43:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:32.659917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ed12-5f64-4612-817c-b3d9c1eda329 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:32.724776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9f89c-55fb-4bd4-b0a8-66d5bbb63bf3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:32.835169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ed12-5f64-4612-817c-b3d9c1eda329 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:32.894296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b71d2-fb55-4b8d-aa38-c06f8b18bcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:32.895265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b71d2-fb55-4b8d-aa38-c06f8b18bcac None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:32.895775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b71d2-fb55-4b8d-aa38-c06f8b18bcac None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:32.896252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b71d2-fb55-4b8d-aa38-c06f8b18bcac None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:32.897130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b71d2-fb55-4b8d-aa38-c06f8b18bcac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:32.898145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b71d2-fb55-4b8d-aa38-c06f8b18bcac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:32.900316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b71d2-fb55-4b8d-aa38-c06f8b18bcac None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:32.905274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9f89c-55fb-4bd4-b0a8-66d5bbb63bf3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:32.915267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 357/716] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Sep 15 10:43:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:32.947975 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe969f0-8b26-4b1b-af7c-1077928a7f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:32.949071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe969f0-8b26-4b1b-af7c-1077928a7f38 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:32.949568 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe969f0-8b26-4b1b-af7c-1077928a7f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:32.950074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe969f0-8b26-4b1b-af7c-1077928a7f38 None admin] PATH_INFO: `/v3/services/network` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:32.950885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe969f0-8b26-4b1b-af7c-1077928a7f38 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:32.953424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe969f0-8b26-4b1b-af7c-1077928a7f38 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3fe969f0-8b26-4b1b-af7c-1077928a7f38 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:32.965708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:32.966759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 10:43:32.968198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:32.976160 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 360/717] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:32 2022] GET /identity/v3/services/network => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:32.977971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b71d2-fb55-4b8d-aa38-c06f8b18bcac None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:33.044014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:33.119057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b71d2-fb55-4b8d-aa38-c06f8b18bcac None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:33.157951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141f045-2ab7-48ae-adf5-9eb2f511ad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:33.158559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141f045-2ab7-48ae-adf5-9eb2f511ad93 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:33.158840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141f045-2ab7-48ae-adf5-9eb2f511ad93 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:33.159106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141f045-2ab7-48ae-adf5-9eb2f511ad93 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:33.159733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141f045-2ab7-48ae-adf5-9eb2f511ad93 None admin] RBAC: Adding query filter params (name=network) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:33.160032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141f045-2ab7-48ae-adf5-9eb2f511ad93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:33.161428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141f045-2ab7-48ae-adf5-9eb2f511ad93 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:33.168915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 358/718] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:43:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:33.175441 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe969f0-8b26-4b1b-af7c-1077928a7f38 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:33.266736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:33.321619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe969f0-8b26-4b1b-af7c-1077928a7f38 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:33.362077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f267aa3-f520-4b71-8c53-4002f154f020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:33.363063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f267aa3-f520-4b71-8c53-4002f154f020 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:33.363561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f267aa3-f520-4b71-8c53-4002f154f020 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:33.364073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f267aa3-f520-4b71-8c53-4002f154f020 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:33.364981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f267aa3-f520-4b71-8c53-4002f154f020 None admin] RBAC: Adding query filter params (type=network) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:33.365492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f267aa3-f520-4b71-8c53-4002f154f020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:33.367100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f267aa3-f520-4b71-8c53-4002f154f020 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:33.380959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 361/719] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:43:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:33.585701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9755a5-52e9-4f50-8d59-992f06aa27ec None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:33.586244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9755a5-52e9-4f50-8d59-992f06aa27ec None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:33.586244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9755a5-52e9-4f50-8d59-992f06aa27ec None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:33.587881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 359/720] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:33.593373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c86e-863c-4e08-8adb-c51788ca78ca None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:33.594324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c86e-863c-4e08-8adb-c51788ca78ca None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:33.594810 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c86e-863c-4e08-8adb-c51788ca78ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:33.621964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b052c86e-863c-4e08-8adb-c51788ca78ca None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:33.624143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b052c86e-863c-4e08-8adb-c51788ca78ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:33.700832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 362/721] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:34.088366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9755a5-52e9-4f50-8d59-992f06aa27ec None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:34.206459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9755a5-52e9-4f50-8d59-992f06aa27ec None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:34.225700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:34.251740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa965c34-bfab-43cb-8f71-e0d7a4fc1b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:34.252385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa965c34-bfab-43cb-8f71-e0d7a4fc1b2a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:34.252665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa965c34-bfab-43cb-8f71-e0d7a4fc1b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:34.252968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa965c34-bfab-43cb-8f71-e0d7a4fc1b2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:34.264500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa965c34-bfab-43cb-8f71-e0d7a4fc1b2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-aa965c34-bfab-43cb-8f71-e0d7a4fc1b2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:34.273603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:34.274737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:34.275595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 360/722] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:34.281520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c86e-863c-4e08-8adb-c51788ca78ca None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:34.374380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:34.440623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c86e-863c-4e08-8adb-c51788ca78ca None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:34.493711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127b0f1-6119-41fa-a716-52156c9e5fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:34.494337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127b0f1-6119-41fa-a716-52156c9e5fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:34.494619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127b0f1-6119-41fa-a716-52156c9e5fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:34.494952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127b0f1-6119-41fa-a716-52156c9e5fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:34.495631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127b0f1-6119-41fa-a716-52156c9e5fc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:34.495935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127b0f1-6119-41fa-a716-52156c9e5fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:34.497408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127b0f1-6119-41fa-a716-52156c9e5fc3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:34.511347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 363/723] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:34.512866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa965c34-bfab-43cb-8f71-e0d7a4fc1b2a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:34.669058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa965c34-bfab-43cb-8f71-e0d7a4fc1b2a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:34.708215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a45a7-adac-4e1a-bddd-529640bf273d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:34.708938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a45a7-adac-4e1a-bddd-529640bf273d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:34.709223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a45a7-adac-4e1a-bddd-529640bf273d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:34.709497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a45a7-adac-4e1a-bddd-529640bf273d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:34.720831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a45a7-adac-4e1a-bddd-529640bf273d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:34.722324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a45a7-adac-4e1a-bddd-529640bf273d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-bf2a45a7-adac-4e1a-bddd-529640bf273d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:34.737868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:34.738872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:34.739873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:34.739873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:34.739873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:34.739873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 361/724] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:34.745148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127b0f1-6119-41fa-a716-52156c9e5fc3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:34.797046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0072e1-9fed-41a1-a0b3-9dddff7cb85f None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:34.797699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0072e1-9fed-41a1-a0b3-9dddff7cb85f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:34.798175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0072e1-9fed-41a1-a0b3-9dddff7cb85f None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:34.800053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 362/725] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:43:34.805936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9131c-5307-4037-a42c-c2e3c87f4b9d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:34.806603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9131c-5307-4037-a42c-c2e3c87f4b9d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:34.807077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9131c-5307-4037-a42c-c2e3c87f4b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:34.827827 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-87c9131c-5307-4037-a42c-c2e3c87f4b9d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:34.829469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-87c9131c-5307-4037-a42c-c2e3c87f4b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:34.877150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:34.938273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127b0f1-6119-41fa-a716-52156c9e5fc3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:34.950685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 363/726] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 10:43:34.952371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074fc24-f53b-4321-afb9-e90efb172412 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:34.952660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074fc24-f53b-4321-afb9-e90efb172412 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:34.952976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074fc24-f53b-4321-afb9-e90efb172412 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:34.958657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 364/727] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:34.959966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ae6f4-68f2-472e-ad2c-0c5fcec16b0d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:34.960249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ae6f4-68f2-472e-ad2c-0c5fcec16b0d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:34.960520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ae6f4-68f2-472e-ad2c-0c5fcec16b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:34.962286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:34.981072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138901c-a123-4f9c-99b7-d077f3d779c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:34.981693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138901c-a123-4f9c-99b7-d077f3d779c5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:34.981974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138901c-a123-4f9c-99b7-d077f3d779c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:34.982246 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138901c-a123-4f9c-99b7-d077f3d779c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:34.983424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138901c-a123-4f9c-99b7-d077f3d779c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:34.983729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138901c-a123-4f9c-99b7-d077f3d779c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:34.984746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-419ae6f4-68f2-472e-ad2c-0c5fcec16b0d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:34.985128 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138901c-a123-4f9c-99b7-d077f3d779c5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:34.986513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-419ae6f4-68f2-472e-ad2c-0c5fcec16b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:34.997476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 364/728] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:35.003444 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138901c-a123-4f9c-99b7-d077f3d779c5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:35.064654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 365/729] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:35.128107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138901c-a123-4f9c-99b7-d077f3d779c5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:35.182875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04773f4-5b5d-48fc-96aa-6ba3e3ba224b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:35.183516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04773f4-5b5d-48fc-96aa-6ba3e3ba224b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:35.183800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04773f4-5b5d-48fc-96aa-6ba3e3ba224b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:35.184074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04773f4-5b5d-48fc-96aa-6ba3e3ba224b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:35.193891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04773f4-5b5d-48fc-96aa-6ba3e3ba224b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:35.195303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04773f4-5b5d-48fc-96aa-6ba3e3ba224b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e04773f4-5b5d-48fc-96aa-6ba3e3ba224b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:35.208003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:35.209104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:35.210094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:35.213157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 365/730] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:35.215152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ae6f4-68f2-472e-ad2c-0c5fcec16b0d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:35.287365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04773f4-5b5d-48fc-96aa-6ba3e3ba224b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:35.368161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:35.369334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:35.402413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ae6f4-68f2-472e-ad2c-0c5fcec16b0d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:35.447183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e00476-15f7-41f6-9fde-37a990229d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:35.447778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e00476-15f7-41f6-9fde-37a990229d75 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:35.448064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e00476-15f7-41f6-9fde-37a990229d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:35.448340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e00476-15f7-41f6-9fde-37a990229d75 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:35.449229 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e00476-15f7-41f6-9fde-37a990229d75 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:35.449537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e00476-15f7-41f6-9fde-37a990229d75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:35.450979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e00476-15f7-41f6-9fde-37a990229d75 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:35.455875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04773f4-5b5d-48fc-96aa-6ba3e3ba224b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:35.470966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 366/731] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Sep 15 10:43:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:35.471882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e00476-15f7-41f6-9fde-37a990229d75 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:35.499497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f947e9e-0b34-4933-9837-a310777502c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:35.500728 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f947e9e-0b34-4933-9837-a310777502c0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:35.501236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f947e9e-0b34-4933-9837-a310777502c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:35.501731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f947e9e-0b34-4933-9837-a310777502c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:35.514696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f947e9e-0b34-4933-9837-a310777502c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5f947e9e-0b34-4933-9837-a310777502c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:35.520253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:35.521377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:35.522429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:35.524781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 366/732] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:35.525933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f947e9e-0b34-4933-9837-a310777502c0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:35.601785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e00476-15f7-41f6-9fde-37a990229d75 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:35.632355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:35.638635 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f947e9e-0b34-4933-9837-a310777502c0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:35.643085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee58dc8-1778-4bdd-a8f9-38de27c5da51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:35.643736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee58dc8-1778-4bdd-a8f9-38de27c5da51 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:35.644082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee58dc8-1778-4bdd-a8f9-38de27c5da51 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:35.644360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee58dc8-1778-4bdd-a8f9-38de27c5da51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:35.656955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee58dc8-1778-4bdd-a8f9-38de27c5da51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-0ee58dc8-1778-4bdd-a8f9-38de27c5da51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:35.661866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:35.662830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:35.663702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 367/733] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:35.664421 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee58dc8-1778-4bdd-a8f9-38de27c5da51 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:35.670936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:35.680084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030a4e0-368c-46d8-9e85-9aec7531abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:35.680700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a4e0-368c-46d8-9e85-9aec7531abdf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:35.681016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a4e0-368c-46d8-9e85-9aec7531abdf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:35.681289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a4e0-368c-46d8-9e85-9aec7531abdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:35.751873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030a4e0-368c-46d8-9e85-9aec7531abdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:35.753309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030a4e0-368c-46d8-9e85-9aec7531abdf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:35.780700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee58dc8-1778-4bdd-a8f9-38de27c5da51 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:35.818767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4904697-adb7-4de7-b00a-c2ebc9a68f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:35.819373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4904697-adb7-4de7-b00a-c2ebc9a68f6e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:35.819652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4904697-adb7-4de7-b00a-c2ebc9a68f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:35.819922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4904697-adb7-4de7-b00a-c2ebc9a68f6e None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:35.820577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4904697-adb7-4de7-b00a-c2ebc9a68f6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:35.820904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4904697-adb7-4de7-b00a-c2ebc9a68f6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:35.822271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4904697-adb7-4de7-b00a-c2ebc9a68f6e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:35.831832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 368/734] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:35.833186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d1a4f-ff08-40bf-9eb8-60bd1c476698 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:35.833803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d1a4f-ff08-40bf-9eb8-60bd1c476698 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:35.833803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d1a4f-ff08-40bf-9eb8-60bd1c476698 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:35.838875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 369/735] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:35.838875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d1a4f-ff08-40bf-9eb8-60bd1c476698 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:35.859856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:35.871372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 367/736] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 10:43:35.872247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030a4e0-368c-46d8-9e85-9aec7531abdf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:35.909719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:35.954982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d1a4f-ff08-40bf-9eb8-60bd1c476698 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:35.975864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030a4e0-368c-46d8-9e85-9aec7531abdf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:35.991614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ff6ae-0076-48f0-9979-d8a468da758f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:35.992218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ff6ae-0076-48f0-9979-d8a468da758f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:35.992710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ff6ae-0076-48f0-9979-d8a468da758f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:35.992999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ff6ae-0076-48f0-9979-d8a468da758f None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:35.993767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ff6ae-0076-48f0-9979-d8a468da758f None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:35.994066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ff6ae-0076-48f0-9979-d8a468da758f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:35.995436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ff6ae-0076-48f0-9979-d8a468da758f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.009401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 370/737] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:36.010038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb3086-daf2-4c74-b585-af20a7e388a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.010507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e69f3-fb5c-45ec-99e3-1e05a6fd5a4c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.010697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3086-daf2-4c74-b585-af20a7e388a0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.010800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e69f3-fb5c-45ec-99e3-1e05a6fd5a4c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.010955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3086-daf2-4c74-b585-af20a7e388a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.011052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e69f3-fb5c-45ec-99e3-1e05a6fd5a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.011213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3086-daf2-4c74-b585-af20a7e388a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.021583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb3086-daf2-4c74-b585-af20a7e388a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.023044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb3086-daf2-4c74-b585-af20a7e388a0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.030353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-116e69f3-fb5c-45ec-99e3-1e05a6fd5a4c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:36.031704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-116e69f3-fb5c-45ec-99e3-1e05a6fd5a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-abeb3086-daf2-4c74-b585-af20a7e388a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:36.038569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:36.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:36.040455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:36.040455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:36.040455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:36.040455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 368/738] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:36.040455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb3086-daf2-4c74-b585-af20a7e388a0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.088853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:36.116848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 371/739] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:36.116848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e69f3-fb5c-45ec-99e3-1e05a6fd5a4c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.166117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb3086-daf2-4c74-b585-af20a7e388a0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:36.194360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:36.216468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a1bd0-0b49-4649-9774-147443d9656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.217218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a1bd0-0b49-4649-9774-147443d9656c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.217641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a1bd0-0b49-4649-9774-147443d9656c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.217981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a1bd0-0b49-4649-9774-147443d9656c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.235549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a1bd0-0b49-4649-9774-147443d9656c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.237047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a1bd0-0b49-4649-9774-147443d9656c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-a74a1bd0-0b49-4649-9774-147443d9656c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:36.252532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:36.253569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:36.254537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:36.254537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:36.254537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:36.258546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 369/740] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:36.259396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a1bd0-0b49-4649-9774-147443d9656c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.261232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e69f3-fb5c-45ec-99e3-1e05a6fd5a4c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:36.302324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ba11a-6989-46b1-b706-caa397ea3bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.303000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ba11a-6989-46b1-b706-caa397ea3bed None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.303284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ba11a-6989-46b1-b706-caa397ea3bed None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.303557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ba11a-6989-46b1-b706-caa397ea3bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.314764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ba11a-6989-46b1-b706-caa397ea3bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.316211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ba11a-6989-46b1-b706-caa397ea3bed None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-7d6ba11a-6989-46b1-b706-caa397ea3bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:36.331353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:36.332322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:36.333290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:36.333290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:36.333290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:36.333290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 372/741] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:36.333290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ba11a-6989-46b1-b706-caa397ea3bed None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.368333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:36.371730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a1bd0-0b49-4649-9774-147443d9656c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:36.407340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c745c0-b5b6-42e7-a78b-af70dbe12809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.407959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c745c0-b5b6-42e7-a78b-af70dbe12809 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.408241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c745c0-b5b6-42e7-a78b-af70dbe12809 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.408519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c745c0-b5b6-42e7-a78b-af70dbe12809 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.409188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c745c0-b5b6-42e7-a78b-af70dbe12809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:36.409491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c745c0-b5b6-42e7-a78b-af70dbe12809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.410877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c745c0-b5b6-42e7-a78b-af70dbe12809 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.423737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 370/742] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:36.425010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49f1f3-db53-4dc8-b629-b4f37493eade None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.425212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49f1f3-db53-4dc8-b629-b4f37493eade None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.425484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49f1f3-db53-4dc8-b629-b4f37493eade None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.428913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 371/743] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:36.428913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49f1f3-db53-4dc8-b629-b4f37493eade None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.442774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ba11a-6989-46b1-b706-caa397ea3bed None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:36.452198 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:36.480185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8877a5-c408-4d0e-8137-7025a48457b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.481105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8877a5-c408-4d0e-8137-7025a48457b1 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.481556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8877a5-c408-4d0e-8137-7025a48457b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.481987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8877a5-c408-4d0e-8137-7025a48457b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.483095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8877a5-c408-4d0e-8137-7025a48457b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:36.483589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8877a5-c408-4d0e-8137-7025a48457b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.485970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8877a5-c408-4d0e-8137-7025a48457b1 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.504132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 373/744] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:36.505181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8877a5-c408-4d0e-8137-7025a48457b1 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.544941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49f1f3-db53-4dc8-b629-b4f37493eade None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:36.582472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3c59-0009-4e84-8171-c14ed06d9ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.583083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3c59-0009-4e84-8171-c14ed06d9ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.583363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3c59-0009-4e84-8171-c14ed06d9ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.583640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3c59-0009-4e84-8171-c14ed06d9ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.584268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a3c59-0009-4e84-8171-c14ed06d9ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:36.584569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a3c59-0009-4e84-8171-c14ed06d9ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.585976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a3c59-0009-4e84-8171-c14ed06d9ee9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.597646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 372/745] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:36.598428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3c59-0009-4e84-8171-c14ed06d9ee9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.611297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8877a5-c408-4d0e-8137-7025a48457b1 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:36.619462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:36.646609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55aed1-9434-473b-9f24-ee3a756ddde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.647225 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55aed1-9434-473b-9f24-ee3a756ddde9 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.647502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55aed1-9434-473b-9f24-ee3a756ddde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.647776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55aed1-9434-473b-9f24-ee3a756ddde9 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.648421 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55aed1-9434-473b-9f24-ee3a756ddde9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.649831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55aed1-9434-473b-9f24-ee3a756ddde9 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.664785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 374/746] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:43:36 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:36.666011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5708-29e3-4de8-9858-bc2f80556bb2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.666300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5708-29e3-4de8-9858-bc2f80556bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.666570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5708-29e3-4de8-9858-bc2f80556bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.686348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-918c5708-29e3-4de8-9858-bc2f80556bb2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:36.687598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-918c5708-29e3-4de8-9858-bc2f80556bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:36.710652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3c59-0009-4e84-8171-c14ed06d9ee9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:36.714212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:36.760840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf12d-3596-4ecf-87f6-cff92b3e8a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.761589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf12d-3596-4ecf-87f6-cff92b3e8a32 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.761823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf12d-3596-4ecf-87f6-cff92b3e8a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.762024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf12d-3596-4ecf-87f6-cff92b3e8a32 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.769838 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 375/747] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 10:43:36.770511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c5708-29e3-4de8-9858-bc2f80556bb2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.778527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cf12d-3596-4ecf-87f6-cff92b3e8a32 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.781744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cf12d-3596-4ecf-87f6-cff92b3e8a32 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-1b2cf12d-3596-4ecf-87f6-cff92b3e8a32 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:36.800622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:36.801695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:36.802680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:36.809693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 373/748] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Sep 15 10:43:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:36.810897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf12d-3596-4ecf-87f6-cff92b3e8a32 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.903638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c5708-29e3-4de8-9858-bc2f80556bb2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:36.905990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:36.937248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf12d-3596-4ecf-87f6-cff92b3e8a32 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:36.949398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9f1a0-592e-4b82-8ece-0fe8ec05be25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.950413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f1a0-592e-4b82-8ece-0fe8ec05be25 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.950775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f1a0-592e-4b82-8ece-0fe8ec05be25 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.951129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f1a0-592e-4b82-8ece-0fe8ec05be25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.959676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9f1a0-592e-4b82-8ece-0fe8ec05be25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.961555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9f1a0-592e-4b82-8ece-0fe8ec05be25 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:36.974210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ba84f-7c0d-4f47-88da-251a43662a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:36.975237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ba84f-7c0d-4f47-88da-251a43662a8b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:36.975661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ba84f-7c0d-4f47-88da-251a43662a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-afb9f1a0-592e-4b82-8ece-0fe8ec05be25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:36.975865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:36.976941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:36.978039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:36.978039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:36.978039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:36.978039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:36.978039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:36.978039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:36.978039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:36.978039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:36.978877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ba84f-7c0d-4f47-88da-251a43662a8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:36.979456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 376/749] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:36.980481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9f1a0-592e-4b82-8ece-0fe8ec05be25 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:36.988613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ba84f-7c0d-4f47-88da-251a43662a8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:36.990432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ba84f-7c0d-4f47-88da-251a43662a8b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:37.002618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-513ba84f-7c0d-4f47-88da-251a43662a8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:37.003543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:37.004493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:43:37.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:37.008174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 374/750] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:43:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:37.009201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ba84f-7c0d-4f47-88da-251a43662a8b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:37.106880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9f1a0-592e-4b82-8ece-0fe8ec05be25 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:37.122138 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ba84f-7c0d-4f47-88da-251a43662a8b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:37.142730 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9c922-bf83-4e22-8dde-6394048b418c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:37.143337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9c922-bf83-4e22-8dde-6394048b418c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:37.143626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9c922-bf83-4e22-8dde-6394048b418c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:37.143885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9c922-bf83-4e22-8dde-6394048b418c None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:37.144599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9c922-bf83-4e22-8dde-6394048b418c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:37.144930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9c922-bf83-4e22-8dde-6394048b418c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:37.146356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9c922-bf83-4e22-8dde-6394048b418c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:37.158789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dd145-4cae-4a81-adac-8a8136c48680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:37.159405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dd145-4cae-4a81-adac-8a8136c48680 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:37.159688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dd145-4cae-4a81-adac-8a8136c48680 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:37.159960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dd145-4cae-4a81-adac-8a8136c48680 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:37.160653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dd145-4cae-4a81-adac-8a8136c48680 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:37.160995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dd145-4cae-4a81-adac-8a8136c48680 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:37.162437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dd145-4cae-4a81-adac-8a8136c48680 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:37.165539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:37.166090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 377/751] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Sep 15 10:43:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:37.166891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9c922-bf83-4e22-8dde-6394048b418c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:37.187080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 375/752] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:37.187080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dd145-4cae-4a81-adac-8a8136c48680 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:37.257544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:37.288532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9c922-bf83-4e22-8dde-6394048b418c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:37.294401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dd145-4cae-4a81-adac-8a8136c48680 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:37.326726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fdfc5-d796-4190-b608-3ab3b9a7e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:37.327491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fdfc5-d796-4190-b608-3ab3b9a7e147 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:37.327603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fdfc5-d796-4190-b608-3ab3b9a7e147 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:37.327882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fdfc5-d796-4190-b608-3ab3b9a7e147 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:37.328905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fdfc5-d796-4190-b608-3ab3b9a7e147 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:37.329306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fdfc5-d796-4190-b608-3ab3b9a7e147 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:37.331244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fdfc5-d796-4190-b608-3ab3b9a7e147 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:37.331627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b902e9-27ca-4697-b079-a4b60e075839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:37.332234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b902e9-27ca-4697-b079-a4b60e075839 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:37.332513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b902e9-27ca-4697-b079-a4b60e075839 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:37.332839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b902e9-27ca-4697-b079-a4b60e075839 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:37.333512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b902e9-27ca-4697-b079-a4b60e075839 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:37.334865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b902e9-27ca-4697-b079-a4b60e075839 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:37.350898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 378/753] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Sep 15 10:43:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:37.351743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fdfc5-d796-4190-b608-3ab3b9a7e147 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:37.352420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 376/754] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:43:37 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:37.353529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b902e9-27ca-4697-b079-a4b60e075839 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:37.434260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:37.499598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b902e9-27ca-4697-b079-a4b60e075839 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:37.520197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fdfc5-d796-4190-b608-3ab3b9a7e147 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:37.547680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973788a0-86dd-4de7-98bd-18ae9bc1bb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:37.549015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973788a0-86dd-4de7-98bd-18ae9bc1bb68 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:37.549936 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973788a0-86dd-4de7-98bd-18ae9bc1bb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:37.550427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973788a0-86dd-4de7-98bd-18ae9bc1bb68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:37.551369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973788a0-86dd-4de7-98bd-18ae9bc1bb68 None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=06007b3b2260409f863ac3d0ac85f4cc) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:37.551908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973788a0-86dd-4de7-98bd-18ae9bc1bb68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:37.554211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973788a0-86dd-4de7-98bd-18ae9bc1bb68 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:37.569916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 377/755] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:37 2022] GET /identity/v3/role_assignments?user.id=06007b3b2260409f863ac3d0ac85f4cc&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=0be9777002c641b4a47f78df647cc173 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:37.569916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973788a0-86dd-4de7-98bd-18ae9bc1bb68 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:37.574731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c7cbe-b5f3-45d4-bc31-e31bbc3d39a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:37.575667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c7cbe-b5f3-45d4-bc31-e31bbc3d39a9 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:37.576137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c7cbe-b5f3-45d4-bc31-e31bbc3d39a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:37.577374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c7cbe-b5f3-45d4-bc31-e31bbc3d39a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:37.580718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:37.600321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c7cbe-b5f3-45d4-bc31-e31bbc3d39a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:37.601848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c7cbe-b5f3-45d4-bc31-e31bbc3d39a9 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-a83c7cbe-b5f3-45d4-bc31-e31bbc3d39a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:37.622404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:37.624025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:37.625087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:37.625087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:37.625087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:37.625087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 379/756] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:37.630806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c7cbe-b5f3-45d4-bc31-e31bbc3d39a9 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:37.735710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973788a0-86dd-4de7-98bd-18ae9bc1bb68 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:37.771901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:37.780087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90e2e6-7ea8-4f40-8ff6-3bf653ab474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:37.780703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90e2e6-7ea8-4f40-8ff6-3bf653ab474a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:37.781016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90e2e6-7ea8-4f40-8ff6-3bf653ab474a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:37.781293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90e2e6-7ea8-4f40-8ff6-3bf653ab474a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:37.809620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c7cbe-b5f3-45d4-bc31-e31bbc3d39a9 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:37.883201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dcb67-b301-4888-b2e4-fed6cab7d911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:37.883814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dcb67-b301-4888-b2e4-fed6cab7d911 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:37.884100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dcb67-b301-4888-b2e4-fed6cab7d911 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:37.884373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dcb67-b301-4888-b2e4-fed6cab7d911 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:37.885055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dcb67-b301-4888-b2e4-fed6cab7d911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:37.885365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dcb67-b301-4888-b2e4-fed6cab7d911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:37.886759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dcb67-b301-4888-b2e4-fed6cab7d911 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:37.908345 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90e2e6-7ea8-4f40-8ff6-3bf653ab474a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:37.909856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90e2e6-7ea8-4f40-8ff6-3bf653ab474a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:37.921238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 380/757] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:37.924490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dcb67-b301-4888-b2e4-fed6cab7d911 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:38.004947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:38.062782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dcb67-b301-4888-b2e4-fed6cab7d911 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:38.085582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 378/758] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:43:38.090954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90e2e6-7ea8-4f40-8ff6-3bf653ab474a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:38.113518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ddd89-e2ad-4c87-90d8-08cb72877b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:38.114171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ddd89-e2ad-4c87-90d8-08cb72877b5e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:38.114459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ddd89-e2ad-4c87-90d8-08cb72877b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:38.114738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ddd89-e2ad-4c87-90d8-08cb72877b5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:38.116248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:38.122257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ddd89-e2ad-4c87-90d8-08cb72877b5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:38.124663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ddd89-e2ad-4c87-90d8-08cb72877b5e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-8f7ddd89-e2ad-4c87-90d8-08cb72877b5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:38.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:38.137713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:38.138745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:38.138745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:38.138745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:38.138745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:38.138745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:38.138745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:38.138745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:38.138745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:38.142645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 381/759] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:38.143891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ddd89-e2ad-4c87-90d8-08cb72877b5e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:38.271720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90e2e6-7ea8-4f40-8ff6-3bf653ab474a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:38.325084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06ea2d-cf90-478a-8841-4738c595b907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:38.325727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06ea2d-cf90-478a-8841-4738c595b907 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:38.326011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06ea2d-cf90-478a-8841-4738c595b907 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:38.326917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06ea2d-cf90-478a-8841-4738c595b907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:38.339460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ddd89-e2ad-4c87-90d8-08cb72877b5e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:38.350607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06ea2d-cf90-478a-8841-4738c595b907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:38.352093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06ea2d-cf90-478a-8841-4738c595b907 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:38.361149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2f06ea2d-cf90-478a-8841-4738c595b907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:38.373597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:38.375737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:38.377865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:38.377865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:38.377865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:38.377865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 379/760] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:38.385782 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06ea2d-cf90-478a-8841-4738c595b907 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:38.408527 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c47e8-ac26-41f5-a1f8-ce4a8dff6d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:38.409213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c47e8-ac26-41f5-a1f8-ce4a8dff6d23 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:38.409492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c47e8-ac26-41f5-a1f8-ce4a8dff6d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:38.409764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c47e8-ac26-41f5-a1f8-ce4a8dff6d23 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:38.410492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c47e8-ac26-41f5-a1f8-ce4a8dff6d23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:38.410802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c47e8-ac26-41f5-a1f8-ce4a8dff6d23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:38.412320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c47e8-ac26-41f5-a1f8-ce4a8dff6d23 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:38.434078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 382/761] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:38.444538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c47e8-ac26-41f5-a1f8-ce4a8dff6d23 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:38.528347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:38.595770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c47e8-ac26-41f5-a1f8-ce4a8dff6d23 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:38.606117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06ea2d-cf90-478a-8841-4738c595b907 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:38.639247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad507e61-9ff0-4308-8aaf-4afad43eb7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:38.640076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad507e61-9ff0-4308-8aaf-4afad43eb7c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:38.640360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad507e61-9ff0-4308-8aaf-4afad43eb7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:38.640630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad507e61-9ff0-4308-8aaf-4afad43eb7c4 None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/218a015dd2b34435a76ecc8a763562ed/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:38.648898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad507e61-9ff0-4308-8aaf-4afad43eb7c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=218a015dd2b34435a76ecc8a763562ed, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:38.650371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad507e61-9ff0-4308-8aaf-4afad43eb7c4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:38.679711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 383/762] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:43:38 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/218a015dd2b34435a76ecc8a763562ed/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:43:38.682142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c15f2-108c-4a58-9fc0-9d3f7a9032c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:38.682768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c15f2-108c-4a58-9fc0-9d3f7a9032c9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:38.683047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c15f2-108c-4a58-9fc0-9d3f7a9032c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:38.683320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c15f2-108c-4a58-9fc0-9d3f7a9032c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:38.683974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c15f2-108c-4a58-9fc0-9d3f7a9032c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:38.684276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c15f2-108c-4a58-9fc0-9d3f7a9032c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:38.685790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c15f2-108c-4a58-9fc0-9d3f7a9032c9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:38.699491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 380/763] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:38.705656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad507e61-9ff0-4308-8aaf-4afad43eb7c4 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:38.740545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:38.817265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad507e61-9ff0-4308-8aaf-4afad43eb7c4 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:38.853995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c6e63-118f-4235-b1dd-7c3d4ee6f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:38.854644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c6e63-118f-4235-b1dd-7c3d4ee6f94f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:38.854925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c6e63-118f-4235-b1dd-7c3d4ee6f94f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:38.855195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c6e63-118f-4235-b1dd-7c3d4ee6f94f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:38.863496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c6e63-118f-4235-b1dd-7c3d4ee6f94f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:38.865107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c6e63-118f-4235-b1dd-7c3d4ee6f94f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-6b4c6e63-118f-4235-b1dd-7c3d4ee6f94f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:38.877347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:38.879462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:38.881602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:38.881602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:38.881602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:38.881602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:38.881602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:38.881602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:38.881602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:38.881602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:38.881602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 384/764] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:38.884548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c15f2-108c-4a58-9fc0-9d3f7a9032c9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:38.999413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c15f2-108c-4a58-9fc0-9d3f7a9032c9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:39.028232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:39.041391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d88232-55e6-4d1c-8d68-3f26d6163357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:39.042034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d88232-55e6-4d1c-8d68-3f26d6163357 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:39.042316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d88232-55e6-4d1c-8d68-3f26d6163357 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:39.042603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d88232-55e6-4d1c-8d68-3f26d6163357 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:39.043310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d88232-55e6-4d1c-8d68-3f26d6163357 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:39.043615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d88232-55e6-4d1c-8d68-3f26d6163357 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:39.045833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d88232-55e6-4d1c-8d68-3f26d6163357 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:39.058536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 381/765] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:39.064586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c6e63-118f-4235-b1dd-7c3d4ee6f94f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:39.121609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d404ae-d493-4eef-bbbe-766266f688d5 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:39.121897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d404ae-d493-4eef-bbbe-766266f688d5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:39.122127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d404ae-d493-4eef-bbbe-766266f688d5 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:39.124038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 382/766] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:39.129913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8739a5-0cf7-419d-a99a-c06ec76af30c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:39.130198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8739a5-0cf7-419d-a99a-c06ec76af30c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:39.130749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8739a5-0cf7-419d-a99a-c06ec76af30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:39.152856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9a8739a5-0cf7-419d-a99a-c06ec76af30c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:39.154078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9a8739a5-0cf7-419d-a99a-c06ec76af30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:39.168479 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:39.187364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c6e63-118f-4235-b1dd-7c3d4ee6f94f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:39.243108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d4144-ee9d-4a11-bf1c-3f320809c833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:39.243723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d4144-ee9d-4a11-bf1c-3f320809c833 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:39.244011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d4144-ee9d-4a11-bf1c-3f320809c833 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:39.244285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d4144-ee9d-4a11-bf1c-3f320809c833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:39.245048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d4144-ee9d-4a11-bf1c-3f320809c833 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=0991767dd90d4f62bfe5484aad435669) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:39.245363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d4144-ee9d-4a11-bf1c-3f320809c833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:39.246802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d4144-ee9d-4a11-bf1c-3f320809c833 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:39.262103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 385/767] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:39 2022] GET /identity/v3/role_assignments?user.id=0991767dd90d4f62bfe5484aad435669&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:39.275890 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 383/768] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:43:39.767790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d4144-ee9d-4a11-bf1c-3f320809c833 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:39.910195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d4144-ee9d-4a11-bf1c-3f320809c833 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:39.945847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba91708-e5e7-4e67-a767-bd8d3341cd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:39.946481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba91708-e5e7-4e67-a767-bd8d3341cd88 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:39.946764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba91708-e5e7-4e67-a767-bd8d3341cd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:39.947042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba91708-e5e7-4e67-a767-bd8d3341cd88 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:39.947633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba91708-e5e7-4e67-a767-bd8d3341cd88 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:39.949071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba91708-e5e7-4e67-a767-bd8d3341cd88 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-2ba91708-e5e7-4e67-a767-bd8d3341cd88 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:39.959306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:39.960390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:39.961294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 386/769] 10.222.0.50 () {56 vars in 1129 bytes} [Thu Sep 15 10:43:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:39.968307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44893e78-c8be-428c-a0df-5c827ba72b6e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:39.968307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44893e78-c8be-428c-a0df-5c827ba72b6e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:39.968307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44893e78-c8be-428c-a0df-5c827ba72b6e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:39.969341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 384/770] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:39.972943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba91708-e5e7-4e67-a767-bd8d3341cd88 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:39.975707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03aeaa5-a1a3-48af-950f-c55ead0e1dfc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:39.976228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03aeaa5-a1a3-48af-950f-c55ead0e1dfc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:39.976976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03aeaa5-a1a3-48af-950f-c55ead0e1dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:39.997199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:39.999804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e03aeaa5-a1a3-48af-950f-c55ead0e1dfc None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:40.001902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e03aeaa5-a1a3-48af-950f-c55ead0e1dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:40.091176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 385/771] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:43:40.150912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba91708-e5e7-4e67-a767-bd8d3341cd88 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:40.199897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050f25a-22dd-48c1-ae96-271ab21c4bcb None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:40.200177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050f25a-22dd-48c1-ae96-271ab21c4bcb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:40.200291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f64a8-e008-4795-bd14-a01a872659eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:40.200388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050f25a-22dd-48c1-ae96-271ab21c4bcb None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:40.201014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f64a8-e008-4795-bd14-a01a872659eb None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:40.201574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f64a8-e008-4795-bd14-a01a872659eb None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:40.201943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f64a8-e008-4795-bd14-a01a872659eb None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:40.202584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f64a8-e008-4795-bd14-a01a872659eb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:40.202886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f64a8-e008-4795-bd14-a01a872659eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:40.203815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 386/772] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:40.204266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f64a8-e008-4795-bd14-a01a872659eb None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:40.208442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58cf9-8f3c-4c77-b4c3-5a976323564b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:40.208813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58cf9-8f3c-4c77-b4c3-5a976323564b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:40.209116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58cf9-8f3c-4c77-b4c3-5a976323564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:40.209869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 387/773] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Sep 15 10:43:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:40.215394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f64a8-e008-4795-bd14-a01a872659eb None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:40.249398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a9f58cf9-8f3c-4c77-b4c3-5a976323564b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:40.255961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a9f58cf9-8f3c-4c77-b4c3-5a976323564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:40.333159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:40.389373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f64a8-e008-4795-bd14-a01a872659eb None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:40.393109 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:40.405251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 387/774] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:40.444273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7397a3-3f08-4e0e-a0d3-7411cddee2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:40.444918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7397a3-3f08-4e0e-a0d3-7411cddee2da None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:40.445237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7397a3-3f08-4e0e-a0d3-7411cddee2da None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:40.445523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7397a3-3f08-4e0e-a0d3-7411cddee2da None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:40.446152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7397a3-3f08-4e0e-a0d3-7411cddee2da None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:40.446506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7397a3-3f08-4e0e-a0d3-7411cddee2da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:40.447932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7397a3-3f08-4e0e-a0d3-7411cddee2da None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:40.453802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 388/775] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Sep 15 10:43:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:40.459546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f58cf9-8f3c-4c77-b4c3-5a976323564b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:40.470427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7397a3-3f08-4e0e-a0d3-7411cddee2da None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:40.648575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f58cf9-8f3c-4c77-b4c3-5a976323564b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:40.657307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7397a3-3f08-4e0e-a0d3-7411cddee2da None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:40.718773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c74d5-e78b-45ab-8c91-90b5cdd72a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:40.719826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c74d5-e78b-45ab-8c91-90b5cdd72a17 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:40.720252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c74d5-e78b-45ab-8c91-90b5cdd72a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:40.720384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cb347-cef4-4178-9e06-fe9e43bdfcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:40.720815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c74d5-e78b-45ab-8c91-90b5cdd72a17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:40.721228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cb347-cef4-4178-9e06-fe9e43bdfcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:40.721606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cb347-cef4-4178-9e06-fe9e43bdfcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:40.721980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cb347-cef4-4178-9e06-fe9e43bdfcc8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:40.722090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c74d5-e78b-45ab-8c91-90b5cdd72a17 None admin] RBAC: Adding query filter params (interface=public, service_id=a717049b6564434697c6d57b11e16ffb, region_id=RegionOne) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:40.722518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c74d5-e78b-45ab-8c91-90b5cdd72a17 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:40.722826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cb347-cef4-4178-9e06-fe9e43bdfcc8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:40.724187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c74d5-e78b-45ab-8c91-90b5cdd72a17 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:40.724339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cb347-cef4-4178-9e06-fe9e43bdfcc8 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-816cb347-cef4-4178-9e06-fe9e43bdfcc8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:40.734525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:40.735670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 10:43:40.736542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:40.737363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 388/776] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Sep 15 10:43:40 2022] GET /identity/v3/endpoints?service_id=a717049b6564434697c6d57b11e16ffb&interface=public®ion_id=RegionOne => generated 194 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:40.738374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c74d5-e78b-45ab-8c91-90b5cdd72a17 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:40.741270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 389/777] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:40 2022] GET /identity/v3/services/network => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:40.742951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cb347-cef4-4178-9e06-fe9e43bdfcc8 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:40.752882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:40.862733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cb347-cef4-4178-9e06-fe9e43bdfcc8 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:40.901060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d3360-cc6e-477c-af8a-51a197ada1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:40.901797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d3360-cc6e-477c-af8a-51a197ada1ac None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:40.902082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d3360-cc6e-477c-af8a-51a197ada1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:40.902354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d3360-cc6e-477c-af8a-51a197ada1ac None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:40.902969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d3360-cc6e-477c-af8a-51a197ada1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:40.905054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d3360-cc6e-477c-af8a-51a197ada1ac None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:40.911019 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 390/778] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Sep 15 10:43:40 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:40.911851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d3360-cc6e-477c-af8a-51a197ada1ac None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:40.930467 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:40.992100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:40.995107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c74d5-e78b-45ab-8c91-90b5cdd72a17 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:41.030931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d3360-cc6e-477c-af8a-51a197ada1ac None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:41.035467 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e87af-350e-466b-a230-41076482b81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:41.036476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e87af-350e-466b-a230-41076482b81e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.037005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e87af-350e-466b-a230-41076482b81e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.037482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e87af-350e-466b-a230-41076482b81e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.050738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e87af-350e-466b-a230-41076482b81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-7b1e87af-350e-466b-a230-41076482b81e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:41.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:41.057249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:41.058135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:41.061804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 389/779] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:41.068189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e87af-350e-466b-a230-41076482b81e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:41.070517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeea43-6948-45fb-83b4-e0a2d04c3f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:41.071314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeea43-6948-45fb-83b4-e0a2d04c3f6e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.071656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeea43-6948-45fb-83b4-e0a2d04c3f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.072006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeea43-6948-45fb-83b4-e0a2d04c3f6e None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.072722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeea43-6948-45fb-83b4-e0a2d04c3f6e None admin] RBAC: Adding query filter params (name=network) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:41.073137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeea43-6948-45fb-83b4-e0a2d04c3f6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:41.074901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeea43-6948-45fb-83b4-e0a2d04c3f6e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:41.081284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 391/780] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:43:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:41.087381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeea43-6948-45fb-83b4-e0a2d04c3f6e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:41.185327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e87af-350e-466b-a230-41076482b81e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:41.189108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:41.231174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeea43-6948-45fb-83b4-e0a2d04c3f6e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:41.269648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a64b5-0f8a-433d-884d-75509c3757c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:41.270176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a64b5-0f8a-433d-884d-75509c3757c4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.270456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a64b5-0f8a-433d-884d-75509c3757c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.270728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a64b5-0f8a-433d-884d-75509c3757c4 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.271358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a64b5-0f8a-433d-884d-75509c3757c4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:41.271469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779567-5836-417d-95b3-cd12da66acf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:41.271668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a64b5-0f8a-433d-884d-75509c3757c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:41.272014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779567-5836-417d-95b3-cd12da66acf0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.272301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779567-5836-417d-95b3-cd12da66acf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.272569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779567-5836-417d-95b3-cd12da66acf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.273059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a64b5-0f8a-433d-884d-75509c3757c4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:41.273236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36779567-5836-417d-95b3-cd12da66acf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:41.273537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36779567-5836-417d-95b3-cd12da66acf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:41.274920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36779567-5836-417d-95b3-cd12da66acf0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:41.276395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:41.279840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 392/781] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:43:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:41.285303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a64b5-0f8a-433d-884d-75509c3757c4 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:41.298337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 390/782] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:41.303962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779567-5836-417d-95b3-cd12da66acf0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:41.444322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a64b5-0f8a-433d-884d-75509c3757c4 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:41.497475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1874b0-8e02-44b9-9fba-bd307350d641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:41.498701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779567-5836-417d-95b3-cd12da66acf0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:41.499214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1874b0-8e02-44b9-9fba-bd307350d641 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.499697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1874b0-8e02-44b9-9fba-bd307350d641 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.500220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1874b0-8e02-44b9-9fba-bd307350d641 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.501221 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1874b0-8e02-44b9-9fba-bd307350d641 None admin] RBAC: Adding query filter params (interface=public, service_id=b2cd9c821efa493a8b99d6ba62fa1a56, region_id=RegionOne) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:41.501746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1874b0-8e02-44b9-9fba-bd307350d641 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:41.503348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1874b0-8e02-44b9-9fba-bd307350d641 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:41.512714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 393/783] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Sep 15 10:43:41 2022] GET /identity/v3/endpoints?service_id=b2cd9c821efa493a8b99d6ba62fa1a56&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:41.514993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59664ad-f07e-451e-a2c1-95ecdcc25d45 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.515494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59664ad-f07e-451e-a2c1-95ecdcc25d45 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.515975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59664ad-f07e-451e-a2c1-95ecdcc25d45 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.521690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 394/784] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:41.522703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59664ad-f07e-451e-a2c1-95ecdcc25d45 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:41.537563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb02a8b-6e4f-4bd1-ba57-034d0ea2a2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:41.538219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb02a8b-6e4f-4bd1-ba57-034d0ea2a2db None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.538546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb02a8b-6e4f-4bd1-ba57-034d0ea2a2db None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.538822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb02a8b-6e4f-4bd1-ba57-034d0ea2a2db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.551616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb02a8b-6e4f-4bd1-ba57-034d0ea2a2db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:41.553234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb02a8b-6e4f-4bd1-ba57-034d0ea2a2db None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0fb02a8b-6e4f-4bd1-ba57-034d0ea2a2db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:41.571730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:41.572823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:41.574374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:41.574374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:41.574374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:41.574374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 391/785] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:41.574617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449f4f8-7d93-4f98-80ca-52746eb0aa94 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.574860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449f4f8-7d93-4f98-80ca-52746eb0aa94 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.575143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449f4f8-7d93-4f98-80ca-52746eb0aa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.596200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2449f4f8-7d93-4f98-80ca-52746eb0aa94 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:41.597459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2449f4f8-7d93-4f98-80ca-52746eb0aa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:41.611228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:41.688832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 392/786] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:41.688832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449f4f8-7d93-4f98-80ca-52746eb0aa94 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:41.745269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:41.799708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449f4f8-7d93-4f98-80ca-52746eb0aa94 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:41.838535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59664ad-f07e-451e-a2c1-95ecdcc25d45 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:41.838989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af079d5f-baa5-427f-ae8a-834f2f1a9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:41.839613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af079d5f-baa5-427f-ae8a-834f2f1a9a6d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.839893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af079d5f-baa5-427f-ae8a-834f2f1a9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.840326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af079d5f-baa5-427f-ae8a-834f2f1a9a6d None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.841002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af079d5f-baa5-427f-ae8a-834f2f1a9a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:41.841305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af079d5f-baa5-427f-ae8a-834f2f1a9a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:41.844331 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af079d5f-baa5-427f-ae8a-834f2f1a9a6d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:41.871968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 393/787] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:41.879815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af079d5f-baa5-427f-ae8a-834f2f1a9a6d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:41.891128 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:41.906736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfd3e4-6907-471a-bab2-32294c758339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:41.907350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfd3e4-6907-471a-bab2-32294c758339 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:41.907623 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfd3e4-6907-471a-bab2-32294c758339 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:41.907887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfd3e4-6907-471a-bab2-32294c758339 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:41.908489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfd3e4-6907-471a-bab2-32294c758339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:41.909874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfd3e4-6907-471a-bab2-32294c758339 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:41.922358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 395/788] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Sep 15 10:43:41 2022] GET /identity/v3/services => generated 772 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:42.021381 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af079d5f-baa5-427f-ae8a-834f2f1a9a6d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:42.039414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfd3e4-6907-471a-bab2-32294c758339 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:42.080950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd20ef-9d86-412b-8878-0265acc6e2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:42.080950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd20ef-9d86-412b-8878-0265acc6e2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.080950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd20ef-9d86-412b-8878-0265acc6e2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.080950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd20ef-9d86-412b-8878-0265acc6e2a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.094598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd20ef-9d86-412b-8878-0265acc6e2a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:42.101547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd20ef-9d86-412b-8878-0265acc6e2a1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b7bd20ef-9d86-412b-8878-0265acc6e2a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:42.121864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:42.123305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:42.124684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:42.129333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 394/789] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:42.137949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd20ef-9d86-412b-8878-0265acc6e2a1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:42.194311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfd3e4-6907-471a-bab2-32294c758339 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:42.210668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:42.226418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:42.241866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c5d3b-8e29-44b5-89a6-6ac7adc03a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:42.242852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c5d3b-8e29-44b5-89a6-6ac7adc03a70 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.243337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c5d3b-8e29-44b5-89a6-6ac7adc03a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.243949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c5d3b-8e29-44b5-89a6-6ac7adc03a70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.254107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd20ef-9d86-412b-8878-0265acc6e2a1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:42.256589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c5d3b-8e29-44b5-89a6-6ac7adc03a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-4e5c5d3b-8e29-44b5-89a6-6ac7adc03a70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:42.261806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:42.262875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:42.264555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:42.267453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 396/790] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:42.269382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640c08d-18d7-4f29-b1e8-2f3092e74f4e None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.269673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640c08d-18d7-4f29-b1e8-2f3092e74f4e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.269948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640c08d-18d7-4f29-b1e8-2f3092e74f4e None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.275946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 397/791] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:42.275946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640c08d-18d7-4f29-b1e8-2f3092e74f4e None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:42.300866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449c814-e6b6-484d-af93-907926e76a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:42.301482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c814-e6b6-484d-af93-907926e76a12 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.301759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c814-e6b6-484d-af93-907926e76a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.302028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c814-e6b6-484d-af93-907926e76a12 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.302751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449c814-e6b6-484d-af93-907926e76a12 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:42.303052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449c814-e6b6-484d-af93-907926e76a12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:42.304490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449c814-e6b6-484d-af93-907926e76a12 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:42.324170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 395/792] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:42.330865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f722fe-2355-412f-a6e1-753e122dbb13 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.331210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f722fe-2355-412f-a6e1-753e122dbb13 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.331425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f722fe-2355-412f-a6e1-753e122dbb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.350198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-45f722fe-2355-412f-a6e1-753e122dbb13 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:42.351392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-45f722fe-2355-412f-a6e1-753e122dbb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:42.384106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640c08d-18d7-4f29-b1e8-2f3092e74f4e None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:42.429683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63ae1e-e77a-42fc-b9d5-62d466668a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:42.430329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63ae1e-e77a-42fc-b9d5-62d466668a6d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.430609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63ae1e-e77a-42fc-b9d5-62d466668a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.431033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63ae1e-e77a-42fc-b9d5-62d466668a6d None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.431706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63ae1e-e77a-42fc-b9d5-62d466668a6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:42.432051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63ae1e-e77a-42fc-b9d5-62d466668a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:42.433686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63ae1e-e77a-42fc-b9d5-62d466668a6d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:42.434742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 396/793] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:42.437306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f722fe-2355-412f-a6e1-753e122dbb13 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:42.448897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 398/794] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:42.452991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63ae1e-e77a-42fc-b9d5-62d466668a6d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:42.500721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:42.509599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:42.551024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f722fe-2355-412f-a6e1-753e122dbb13 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:42.573023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63ae1e-e77a-42fc-b9d5-62d466668a6d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:42.590280 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4a6ad-ae50-4224-b9f3-119a10489285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:42.590853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4a6ad-ae50-4224-b9f3-119a10489285 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.591187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4a6ad-ae50-4224-b9f3-119a10489285 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.591552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4a6ad-ae50-4224-b9f3-119a10489285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.602494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4a6ad-ae50-4224-b9f3-119a10489285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:42.603947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4a6ad-ae50-4224-b9f3-119a10489285 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:42.615011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e05a38-2606-40ac-8c20-04ced7219736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:42.615640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e05a38-2606-40ac-8c20-04ced7219736 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.615915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e05a38-2606-40ac-8c20-04ced7219736 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.616183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e05a38-2606-40ac-8c20-04ced7219736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c3f4a6ad-ae50-4224-b9f3-119a10489285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:42.620326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:42.621287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:42.622219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:42.622219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:42.622219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:42.622219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 397/795] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:42.636078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4a6ad-ae50-4224-b9f3-119a10489285 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:42.637707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e05a38-2606-40ac-8c20-04ced7219736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:42.639629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e05a38-2606-40ac-8c20-04ced7219736 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-c5e05a38-2606-40ac-8c20-04ced7219736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:42.657025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:42.659091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:42.661059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:42.661059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:42.661059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:42.661059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 399/796] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:42.670324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e05a38-2606-40ac-8c20-04ced7219736 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:42.761070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4a6ad-ae50-4224-b9f3-119a10489285 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:42.769415 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:42.809522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb1792-f718-46be-927b-2913ff5ac2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:42.811023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb1792-f718-46be-927b-2913ff5ac2ab None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.811486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb1792-f718-46be-927b-2913ff5ac2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.811910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb1792-f718-46be-927b-2913ff5ac2ab None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.812714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb1792-f718-46be-927b-2913ff5ac2ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:42.813066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb1792-f718-46be-927b-2913ff5ac2ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:42.814448 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb1792-f718-46be-927b-2913ff5ac2ab None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:42.828031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 398/797] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:42.828973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb1792-f718-46be-927b-2913ff5ac2ab None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:42.834973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:42.839149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e05a38-2606-40ac-8c20-04ced7219736 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:42.878206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaa346-dc4b-46ff-8c95-442a809f9abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:42.879650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaa346-dc4b-46ff-8c95-442a809f9abc None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.880390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaa346-dc4b-46ff-8c95-442a809f9abc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.880918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaa346-dc4b-46ff-8c95-442a809f9abc None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.881860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faaa346-dc4b-46ff-8c95-442a809f9abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:42.882421 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faaa346-dc4b-46ff-8c95-442a809f9abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:42.884006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faaa346-dc4b-46ff-8c95-442a809f9abc None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:42.899052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 400/798] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:42.900191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaa346-dc4b-46ff-8c95-442a809f9abc None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:42.941589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb1792-f718-46be-927b-2913ff5ac2ab None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:42.977151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e544e9d-ff69-425b-89e7-5c184b7e8ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:42.977820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544e9d-ff69-425b-89e7-5c184b7e8ffd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:42.978103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544e9d-ff69-425b-89e7-5c184b7e8ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:42.978373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544e9d-ff69-425b-89e7-5c184b7e8ffd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:42.990068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e544e9d-ff69-425b-89e7-5c184b7e8ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-8e544e9d-ff69-425b-89e7-5c184b7e8ffd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:42.995488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:42.996461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:42.997363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 399/799] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:42.998644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e544e9d-ff69-425b-89e7-5c184b7e8ffd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:43.012751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaa346-dc4b-46ff-8c95-442a809f9abc None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:43.028212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:43.053529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8a9ae-9166-4e67-a4fb-08dbedd732d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:43.054500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8a9ae-9166-4e67-a4fb-08dbedd732d5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.054971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8a9ae-9166-4e67-a4fb-08dbedd732d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.055378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8a9ae-9166-4e67-a4fb-08dbedd732d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.063838 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8a9ae-9166-4e67-a4fb-08dbedd732d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:43.065595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8a9ae-9166-4e67-a4fb-08dbedd732d5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-98f8a9ae-9166-4e67-a4fb-08dbedd732d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:43.078507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:43.079449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:43.080368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:43.080368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:43.080368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:43.080368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:43.080368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:43.080368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:43.080368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:43.080368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:43.087636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 401/800] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:43.088708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8a9ae-9166-4e67-a4fb-08dbedd732d5 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:43.106000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e544e9d-ff69-425b-89e7-5c184b7e8ffd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:43.141829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:43.145038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001364c1-8744-4bd3-8ed5-0d154d4ccff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:43.146039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001364c1-8744-4bd3-8ed5-0d154d4ccff5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.146521 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001364c1-8744-4bd3-8ed5-0d154d4ccff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.147103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001364c1-8744-4bd3-8ed5-0d154d4ccff5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.156946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001364c1-8744-4bd3-8ed5-0d154d4ccff5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:43.158803 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001364c1-8744-4bd3-8ed5-0d154d4ccff5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-001364c1-8744-4bd3-8ed5-0d154d4ccff5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:43.173105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:43.174144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:43:43.175118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:43.182409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 400/801] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Sep 15 10:43:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:43.183623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001364c1-8744-4bd3-8ed5-0d154d4ccff5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:43.207066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8a9ae-9166-4e67-a4fb-08dbedd732d5 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:43.244571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951316e-0ff0-4e80-83d8-33130938ac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:43.245204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951316e-0ff0-4e80-83d8-33130938ac9c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.245482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951316e-0ff0-4e80-83d8-33130938ac9c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.245752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951316e-0ff0-4e80-83d8-33130938ac9c None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.246511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951316e-0ff0-4e80-83d8-33130938ac9c None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:43.246813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951316e-0ff0-4e80-83d8-33130938ac9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:43.249207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951316e-0ff0-4e80-83d8-33130938ac9c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:43.258441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 402/802] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:43.259304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951316e-0ff0-4e80-83d8-33130938ac9c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:43.285218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:43.306405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001364c1-8744-4bd3-8ed5-0d154d4ccff5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:43.344299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e8ad2-0cd1-4a31-b86e-7a24eeb0fedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:43.345344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e8ad2-0cd1-4a31-b86e-7a24eeb0fedc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.345836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e8ad2-0cd1-4a31-b86e-7a24eeb0fedc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.346363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e8ad2-0cd1-4a31-b86e-7a24eeb0fedc None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.347248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e8ad2-0cd1-4a31-b86e-7a24eeb0fedc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:43.347776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e8ad2-0cd1-4a31-b86e-7a24eeb0fedc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:43.349651 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e8ad2-0cd1-4a31-b86e-7a24eeb0fedc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:43.353276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:43.366374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 401/803] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:43.367628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e8ad2-0cd1-4a31-b86e-7a24eeb0fedc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:43.379283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951316e-0ff0-4e80-83d8-33130938ac9c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:43.416660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9afe9-05da-4bb7-a331-587f3f5a4f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:43.417296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9afe9-05da-4bb7-a331-587f3f5a4f5b None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.417573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9afe9-05da-4bb7-a331-587f3f5a4f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.417843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9afe9-05da-4bb7-a331-587f3f5a4f5b None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.418587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9afe9-05da-4bb7-a331-587f3f5a4f5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:43.418852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9afe9-05da-4bb7-a331-587f3f5a4f5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:43.421207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9afe9-05da-4bb7-a331-587f3f5a4f5b None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:43.439193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 403/804] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Sep 15 10:43:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:43.439803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9afe9-05da-4bb7-a331-587f3f5a4f5b None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:43.479523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e8ad2-0cd1-4a31-b86e-7a24eeb0fedc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:43.518611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eef9a8-ffaa-4496-8223-0f60ad796947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:43.519243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eef9a8-ffaa-4496-8223-0f60ad796947 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.519525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eef9a8-ffaa-4496-8223-0f60ad796947 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.519794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eef9a8-ffaa-4496-8223-0f60ad796947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.530663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eef9a8-ffaa-4496-8223-0f60ad796947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:43.532124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eef9a8-ffaa-4496-8223-0f60ad796947 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:43.543040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-59eef9a8-ffaa-4496-8223-0f60ad796947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:43.551154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:43.552344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:43.553507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:43.553507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:43.553507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:43.554026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9afe9-05da-4bb7-a331-587f3f5a4f5b None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:43.558235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 402/805] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:43.559542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6113d8-f4f2-4a8d-abf1-7665fd445844 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.559834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6113d8-f4f2-4a8d-abf1-7665fd445844 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.560669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6113d8-f4f2-4a8d-abf1-7665fd445844 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.562680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 403/806] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:43.563762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6113d8-f4f2-4a8d-abf1-7665fd445844 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:43.604744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95f734-4a14-4baf-bce6-e153d4cbdd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:43.605601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95f734-4a14-4baf-bce6-e153d4cbdd8a None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.605983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95f734-4a14-4baf-bce6-e153d4cbdd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.606357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95f734-4a14-4baf-bce6-e153d4cbdd8a None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/06007b3b2260409f863ac3d0ac85f4cc/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.608427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95f734-4a14-4baf-bce6-e153d4cbdd8a None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=06007b3b2260409f863ac3d0ac85f4cc, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:43.610289 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95f734-4a14-4baf-bce6-e153d4cbdd8a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:43.619513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:43.632844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 404/807] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:43:43 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/06007b3b2260409f863ac3d0ac85f4cc/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:43:43.632844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95f734-4a14-4baf-bce6-e153d4cbdd8a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:43.692956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6113d8-f4f2-4a8d-abf1-7665fd445844 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:43.739481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720780b-2f1c-490f-8358-b2cf5a244b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:43.740107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720780b-2f1c-490f-8358-b2cf5a244b9c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.740461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720780b-2f1c-490f-8358-b2cf5a244b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.740818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720780b-2f1c-490f-8358-b2cf5a244b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.762797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95f734-4a14-4baf-bce6-e153d4cbdd8a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:43.806143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e1fe8-f8b7-4c63-935a-738e5f38a514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:43.807514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e1fe8-f8b7-4c63-935a-738e5f38a514 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.808203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e1fe8-f8b7-4c63-935a-738e5f38a514 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.808622 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e1fe8-f8b7-4c63-935a-738e5f38a514 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.809557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e1fe8-f8b7-4c63-935a-738e5f38a514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:43.810196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e1fe8-f8b7-4c63-935a-738e5f38a514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:43.812475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e1fe8-f8b7-4c63-935a-738e5f38a514 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:43.818134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:43.821964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720780b-2f1c-490f-8358-b2cf5a244b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:43.824448 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720780b-2f1c-490f-8358-b2cf5a244b9c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:43.830035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 405/808] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:43.831470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be2575-d51a-4dc6-abdb-b7db815c2984 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:43.831918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be2575-d51a-4dc6-abdb-b7db815c2984 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:43.832336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be2575-d51a-4dc6-abdb-b7db815c2984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:43.858681 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-b4be2575-d51a-4dc6-abdb-b7db815c2984 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:43.859905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b4be2575-d51a-4dc6-abdb-b7db815c2984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:43.928495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:43.940969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 406/809] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:43.942153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be2575-d51a-4dc6-abdb-b7db815c2984 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:43.973616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 404/810] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:43:43.979345 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720780b-2f1c-490f-8358-b2cf5a244b9c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:44.097296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be2575-d51a-4dc6-abdb-b7db815c2984 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:44.152188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb0d64-34a7-4a6a-a4b6-f6f6408233af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:44.152847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb0d64-34a7-4a6a-a4b6-f6f6408233af None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.153186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb0d64-34a7-4a6a-a4b6-f6f6408233af None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.153461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb0d64-34a7-4a6a-a4b6-f6f6408233af None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.161753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb0d64-34a7-4a6a-a4b6-f6f6408233af None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:44.163489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb0d64-34a7-4a6a-a4b6-f6f6408233af None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:44.174392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-2dbb0d64-34a7-4a6a-a4b6-f6f6408233af None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:44.177898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:44.179233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:43:44.180223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:44.180780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720780b-2f1c-490f-8358-b2cf5a244b9c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:44.182650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 407/811] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:43:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:44.188849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb0d64-34a7-4a6a-a4b6-f6f6408233af None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:44.246284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1141a3-faa4-495e-a581-f220f1d98476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:44.246845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1141a3-faa4-495e-a581-f220f1d98476 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.247119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1141a3-faa4-495e-a581-f220f1d98476 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.247390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1141a3-faa4-495e-a581-f220f1d98476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.270025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1141a3-faa4-495e-a581-f220f1d98476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:44.271493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1141a3-faa4-495e-a581-f220f1d98476 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:44.288471 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-1a1141a3-faa4-495e-a581-f220f1d98476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:44.298701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:44.299711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:44.300629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:44.300629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:44.300629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:44.300629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 405/812] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:44.300897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1141a3-faa4-495e-a581-f220f1d98476 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:44.352058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb0d64-34a7-4a6a-a4b6-f6f6408233af None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:44.396925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585920fb-e60f-4af8-8e8c-8bc18356c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:44.397586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585920fb-e60f-4af8-8e8c-8bc18356c97c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.397870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585920fb-e60f-4af8-8e8c-8bc18356c97c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.398145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585920fb-e60f-4af8-8e8c-8bc18356c97c None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.399609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585920fb-e60f-4af8-8e8c-8bc18356c97c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:44.399929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585920fb-e60f-4af8-8e8c-8bc18356c97c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:44.401739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585920fb-e60f-4af8-8e8c-8bc18356c97c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:44.413229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1141a3-faa4-495e-a581-f220f1d98476 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:44.428075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 408/813] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Sep 15 10:43:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:44.429038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585920fb-e60f-4af8-8e8c-8bc18356c97c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:44.455867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d0840-2034-4cbf-a850-c2439ed0e919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:44.456502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d0840-2034-4cbf-a850-c2439ed0e919 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.456779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d0840-2034-4cbf-a850-c2439ed0e919 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.457096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d0840-2034-4cbf-a850-c2439ed0e919 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.457675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d0840-2034-4cbf-a850-c2439ed0e919 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:44.459047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d0840-2034-4cbf-a850-c2439ed0e919 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-fb1d0840-2034-4cbf-a850-c2439ed0e919 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:44.469740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:44.472024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:44.473913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 406/814] 10.222.0.50 () {56 vars in 1129 bytes} [Thu Sep 15 10:43:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:44.475748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d0840-2034-4cbf-a850-c2439ed0e919 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:44.489963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:44.567665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585920fb-e60f-4af8-8e8c-8bc18356c97c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:44.579163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:44.583940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d0840-2034-4cbf-a850-c2439ed0e919 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:44.606403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5372c-7960-459d-a868-3e3f68ef34d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:44.607651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5372c-7960-459d-a868-3e3f68ef34d5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.607939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5372c-7960-459d-a868-3e3f68ef34d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.608215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5372c-7960-459d-a868-3e3f68ef34d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.608894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5372c-7960-459d-a868-3e3f68ef34d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:44.609201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5372c-7960-459d-a868-3e3f68ef34d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:44.610621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5372c-7960-459d-a868-3e3f68ef34d5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:44.620316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f4f9c-5184-48e3-b271-f34509967108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:44.620968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f4f9c-5184-48e3-b271-f34509967108 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.621247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f4f9c-5184-48e3-b271-f34509967108 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.621517 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f4f9c-5184-48e3-b271-f34509967108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.622418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 409/815] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:44.623910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5372c-7960-459d-a868-3e3f68ef34d5 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:44.639459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f4f9c-5184-48e3-b271-f34509967108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:44.640992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f4f9c-5184-48e3-b271-f34509967108 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-132f4f9c-5184-48e3-b271-f34509967108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:44.657828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:44.658875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:44.659792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:44.659792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:44.659792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:44.659792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 407/816] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:44.660473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e88a-9710-44e9-80b1-5c06ed60ec62 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.660570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e88a-9710-44e9-80b1-5c06ed60ec62 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.660835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e88a-9710-44e9-80b1-5c06ed60ec62 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.662214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 408/817] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:44.665575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e88a-9710-44e9-80b1-5c06ed60ec62 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:44.736343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:44.745018 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5372c-7960-459d-a868-3e3f68ef34d5 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:44.774083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e88a-9710-44e9-80b1-5c06ed60ec62 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:44.782592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de97f4c-35fd-4f13-ade9-7db55d9f4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:44.783198 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de97f4c-35fd-4f13-ade9-7db55d9f4e79 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.783599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de97f4c-35fd-4f13-ade9-7db55d9f4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.783877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de97f4c-35fd-4f13-ade9-7db55d9f4e79 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.784512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de97f4c-35fd-4f13-ade9-7db55d9f4e79 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:44.784864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de97f4c-35fd-4f13-ade9-7db55d9f4e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:44.786311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de97f4c-35fd-4f13-ade9-7db55d9f4e79 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:44.790778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 410/818] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Sep 15 10:43:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:44.791553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de97f4c-35fd-4f13-ade9-7db55d9f4e79 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:44.810236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78070d1-34e9-4251-ac34-f32ce708ae79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:44.811094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78070d1-34e9-4251-ac34-f32ce708ae79 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.811375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78070d1-34e9-4251-ac34-f32ce708ae79 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.811650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78070d1-34e9-4251-ac34-f32ce708ae79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.818340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78070d1-34e9-4251-ac34-f32ce708ae79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:44.819791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78070d1-34e9-4251-ac34-f32ce708ae79 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:44.827545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c78070d1-34e9-4251-ac34-f32ce708ae79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:44.832434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:44.833360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:44.834266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:44.834266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:44.834266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:44.834266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:44.834266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:44.834266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:44.834266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:44.834266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:44.838118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 409/819] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:44.839325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4a96e-654d-48c1-9430-4da5cc2eb1b1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.839610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4a96e-654d-48c1-9430-4da5cc2eb1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.839883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4a96e-654d-48c1-9430-4da5cc2eb1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.859216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4bf4a96e-654d-48c1-9430-4da5cc2eb1b1 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:44.860423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4bf4a96e-654d-48c1-9430-4da5cc2eb1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:44.904167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de97f4c-35fd-4f13-ade9-7db55d9f4e79 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:44.929582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 410/820] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:44.930428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4a96e-654d-48c1-9430-4da5cc2eb1b1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:44.947740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a648968-9409-4f36-8392-2d499cacded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:44.948387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a648968-9409-4f36-8392-2d499cacded3 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:44.948666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a648968-9409-4f36-8392-2d499cacded3 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:44.948978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a648968-9409-4f36-8392-2d499cacded3 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:44.949670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a648968-9409-4f36-8392-2d499cacded3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:44.949966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a648968-9409-4f36-8392-2d499cacded3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:44.952539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a648968-9409-4f36-8392-2d499cacded3 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:44.971112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 411/821] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:44.972002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a648968-9409-4f36-8392-2d499cacded3 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.017082 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:45.080613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4a96e-654d-48c1-9430-4da5cc2eb1b1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.086670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:45.099458 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a648968-9409-4f36-8392-2d499cacded3 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.125761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf1c66-ee1b-46cd-a24f-bd1e158208ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.126671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf1c66-ee1b-46cd-a24f-bd1e158208ac None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.127384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf1c66-ee1b-46cd-a24f-bd1e158208ac None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.127875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf1c66-ee1b-46cd-a24f-bd1e158208ac None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.128909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf1c66-ee1b-46cd-a24f-bd1e158208ac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:45.129731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf1c66-ee1b-46cd-a24f-bd1e158208ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.131586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf1c66-ee1b-46cd-a24f-bd1e158208ac None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.138794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 411/822] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Sep 15 10:43:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:45.139884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf1c66-ee1b-46cd-a24f-bd1e158208ac None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.146060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6e7e-8a7b-4025-b636-af54f0f47f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.146750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6e7e-8a7b-4025-b636-af54f0f47f9f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.147113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6e7e-8a7b-4025-b636-af54f0f47f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.147449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6e7e-8a7b-4025-b636-af54f0f47f9f None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.148243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf6e7e-8a7b-4025-b636-af54f0f47f9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:45.148624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf6e7e-8a7b-4025-b636-af54f0f47f9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.150515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf6e7e-8a7b-4025-b636-af54f0f47f9f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.164849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 412/823] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:45.165734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6e7e-8a7b-4025-b636-af54f0f47f9f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.258730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf1c66-ee1b-46cd-a24f-bd1e158208ac None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.276936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6e7e-8a7b-4025-b636-af54f0f47f9f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.281247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:45.296305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513662e-9673-4a90-976c-b1136f855c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.297039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513662e-9673-4a90-976c-b1136f855c4d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.297319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513662e-9673-4a90-976c-b1136f855c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.297589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513662e-9673-4a90-976c-b1136f855c4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.304186 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513662e-9673-4a90-976c-b1136f855c4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.305658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513662e-9673-4a90-976c-b1136f855c4d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.315561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a9fee-0d48-48ca-a975-898b2f9d71cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.316415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a9fee-0d48-48ca-a975-898b2f9d71cc None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.316914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a9fee-0d48-48ca-a975-898b2f9d71cc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b513662e-9673-4a90-976c-b1136f855c4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:45.317255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:45.318150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 412/824] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:45.319075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513662e-9673-4a90-976c-b1136f855c4d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.320340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a9fee-0d48-48ca-a975-898b2f9d71cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.321064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a9fee-0d48-48ca-a975-898b2f9d71cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.322455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a9fee-0d48-48ca-a975-898b2f9d71cc None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.349017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 413/825] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:43:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:45.349605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a9fee-0d48-48ca-a975-898b2f9d71cc None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.362867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:45.469744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513662e-9673-4a90-976c-b1136f855c4d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.489221 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a9fee-0d48-48ca-a975-898b2f9d71cc None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.510471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05a489-8e2d-4567-a4be-1abbe7f3b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.511474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05a489-8e2d-4567-a4be-1abbe7f3b10d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.511953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05a489-8e2d-4567-a4be-1abbe7f3b10d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.512442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05a489-8e2d-4567-a4be-1abbe7f3b10d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.513545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05a489-8e2d-4567-a4be-1abbe7f3b10d None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=218a015dd2b34435a76ecc8a763562ed) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:45.514356 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05a489-8e2d-4567-a4be-1abbe7f3b10d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.516052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05a489-8e2d-4567-a4be-1abbe7f3b10d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.529154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 413/826] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:45 2022] GET /identity/v3/role_assignments?user.id=218a015dd2b34435a76ecc8a763562ed&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=0be9777002c641b4a47f78df647cc173 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:45.529945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05a489-8e2d-4567-a4be-1abbe7f3b10d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.534328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae772f-6c63-4638-bd6e-9a43bad3d149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.534986 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae772f-6c63-4638-bd6e-9a43bad3d149 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.535268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae772f-6c63-4638-bd6e-9a43bad3d149 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.535601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae772f-6c63-4638-bd6e-9a43bad3d149 None admin] PATH_INFO: `/v3/services/network` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.536204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae772f-6c63-4638-bd6e-9a43bad3d149 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.537766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae772f-6c63-4638-bd6e-9a43bad3d149 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-03ae772f-6c63-4638-bd6e-9a43bad3d149 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:45.550659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:45.551812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:45.552763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 414/827] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:45 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:45.560961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae772f-6c63-4638-bd6e-9a43bad3d149 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.593162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:45.639634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:45.655535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05a489-8e2d-4567-a4be-1abbe7f3b10d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.680498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae772f-6c63-4638-bd6e-9a43bad3d149 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.695427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c9af0-a4bc-4bad-be26-640e4f48298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.696052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c9af0-a4bc-4bad-be26-640e4f48298e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.696333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c9af0-a4bc-4bad-be26-640e4f48298e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.696607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c9af0-a4bc-4bad-be26-640e4f48298e None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.697362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c9af0-a4bc-4bad-be26-640e4f48298e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:45.697670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c9af0-a4bc-4bad-be26-640e4f48298e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.699142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c9af0-a4bc-4bad-be26-640e4f48298e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.713227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 414/828] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:45.719752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c9af0-a4bc-4bad-be26-640e4f48298e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.721425 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11774f27-ad7e-412e-97cd-83a05b1cac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.722049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11774f27-ad7e-412e-97cd-83a05b1cac58 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.722367 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11774f27-ad7e-412e-97cd-83a05b1cac58 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.722785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11774f27-ad7e-412e-97cd-83a05b1cac58 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.723553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11774f27-ad7e-412e-97cd-83a05b1cac58 None admin] RBAC: Adding query filter params (name=network) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:45.724005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11774f27-ad7e-412e-97cd-83a05b1cac58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.725592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11774f27-ad7e-412e-97cd-83a05b1cac58 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.731334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 415/829] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:43:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:45.739694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11774f27-ad7e-412e-97cd-83a05b1cac58 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.838035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c9af0-a4bc-4bad-be26-640e4f48298e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.854092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:45.857974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11774f27-ad7e-412e-97cd-83a05b1cac58 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:45.893762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d73a4-928d-4394-822f-0c4569a7ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.894568 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d73a4-928d-4394-822f-0c4569a7ccb3 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.894948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d73a4-928d-4394-822f-0c4569a7ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.895329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d73a4-928d-4394-822f-0c4569a7ccb3 None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/0991767dd90d4f62bfe5484aad435669/roles/fa23cf1cb845457d9460c2a9884af4cf` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.899394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:45.907541 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775c459-b8f0-48b2-9efc-ebdb0a4e16ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:45.908255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775c459-b8f0-48b2-9efc-ebdb0a4e16ef None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:45.908536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775c459-b8f0-48b2-9efc-ebdb0a4e16ef None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:45.908833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775c459-b8f0-48b2-9efc-ebdb0a4e16ef None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:45.909482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775c459-b8f0-48b2-9efc-ebdb0a4e16ef None admin] RBAC: Adding query filter params (type=network) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:45.909784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775c459-b8f0-48b2-9efc-ebdb0a4e16ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.911211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775c459-b8f0-48b2-9efc-ebdb0a4e16ef None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.917090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d73a4-928d-4394-822f-0c4569a7ccb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=0991767dd90d4f62bfe5484aad435669, role_id=fa23cf1cb845457d9460c2a9884af4cf)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:45.918582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 416/830] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:43:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:45.919153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d73a4-928d-4394-822f-0c4569a7ccb3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:45.926553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775c459-b8f0-48b2-9efc-ebdb0a4e16ef None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:45.944493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 415/831] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:43:45 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/0991767dd90d4f62bfe5484aad435669/roles/fa23cf1cb845457d9460c2a9884af4cf => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:43:46.025604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdec0c-cd58-446b-a4dd-ab72c8b068a1 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:46.025891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdec0c-cd58-446b-a4dd-ab72c8b068a1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:46.026170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdec0c-cd58-446b-a4dd-ab72c8b068a1 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:46.029854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 416/832] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:43:46.034551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bec3-bcd4-45bd-b4f3-3a16e271f467 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:46.034833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bec3-bcd4-45bd-b4f3-3a16e271f467 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:46.035119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bec3-bcd4-45bd-b4f3-3a16e271f467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:46.053638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775c459-b8f0-48b2-9efc-ebdb0a4e16ef None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:46.057961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e7e6bec3-bcd4-45bd-b4f3-3a16e271f467 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:46.059211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e7e6bec3-bcd4-45bd-b4f3-3a16e271f467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:46.096562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4ce3a-0e44-449b-9999-b5b98f274a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:46.097469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4ce3a-0e44-449b-9999-b5b98f274a9f None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:46.097905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4ce3a-0e44-449b-9999-b5b98f274a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:46.098328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4ce3a-0e44-449b-9999-b5b98f274a9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:46.099170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4ce3a-0e44-449b-9999-b5b98f274a9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:46.100921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4ce3a-0e44-449b-9999-b5b98f274a9f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:46.132311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 417/833] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:43:45 2022] POST /identity/v3/endpoints => generated 337 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:46.194975 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 417/834] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:43:46.489693 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4ce3a-0e44-449b-9999-b5b98f274a9f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:46.554958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:46.614089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4ce3a-0e44-449b-9999-b5b98f274a9f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:46.657431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b54ec-6efa-49e5-bc02-45a35f06b948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:46.658071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b54ec-6efa-49e5-bc02-45a35f06b948 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:46.658348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b54ec-6efa-49e5-bc02-45a35f06b948 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:46.658614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b54ec-6efa-49e5-bc02-45a35f06b948 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:46.672859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b54ec-6efa-49e5-bc02-45a35f06b948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-f05b54ec-6efa-49e5-bc02-45a35f06b948 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:46.678158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:46.679089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:46.679981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 418/835] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:46.686267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bec3-bcd4-45bd-b4f3-3a16e271f467 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:46.778860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:46.801159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bec3-bcd4-45bd-b4f3-3a16e271f467 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:46.840276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d43f43-6f15-4893-a14f-dcbfe6ddb96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:46.840913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d43f43-6f15-4893-a14f-dcbfe6ddb96c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:46.841322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d43f43-6f15-4893-a14f-dcbfe6ddb96c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:46.841752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d43f43-6f15-4893-a14f-dcbfe6ddb96c None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:46.842610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d43f43-6f15-4893-a14f-dcbfe6ddb96c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:46.842911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d43f43-6f15-4893-a14f-dcbfe6ddb96c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:46.844350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d43f43-6f15-4893-a14f-dcbfe6ddb96c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:46.853275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 418/836] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:43:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:46.860106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b54ec-6efa-49e5-bc02-45a35f06b948 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:46.970414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b54ec-6efa-49e5-bc02-45a35f06b948 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:46.984254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:47.006206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3f074-633c-4258-b101-c22ec057420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:47.006847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3f074-633c-4258-b101-c22ec057420a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:47.007125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3f074-633c-4258-b101-c22ec057420a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:47.007397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3f074-633c-4258-b101-c22ec057420a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:47.019649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3f074-633c-4258-b101-c22ec057420a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:47.021149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3f074-633c-4258-b101-c22ec057420a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-2bf3f074-633c-4258-b101-c22ec057420a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:47.036780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:47.037771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:47.038685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:47.038685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:47.038685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:47.038685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 419/837] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:47.045423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d43f43-6f15-4893-a14f-dcbfe6ddb96c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:47.154951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d43f43-6f15-4893-a14f-dcbfe6ddb96c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:47.190595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd6226-6956-4bf0-bb7a-e612799d6b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:47.191217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd6226-6956-4bf0-bb7a-e612799d6b99 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:47.191490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd6226-6956-4bf0-bb7a-e612799d6b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:47.191764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd6226-6956-4bf0-bb7a-e612799d6b99 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:47.192406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd6226-6956-4bf0-bb7a-e612799d6b99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:47.192760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd6226-6956-4bf0-bb7a-e612799d6b99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:47.194166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd6226-6956-4bf0-bb7a-e612799d6b99 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:47.208372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 419/838] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:47.213790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3f074-633c-4258-b101-c22ec057420a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:47.328246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3f074-633c-4258-b101-c22ec057420a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:47.363816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e1828-b9a9-4218-a8a7-c439a48e1421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:47.364454 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e1828-b9a9-4218-a8a7-c439a48e1421 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:47.364734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e1828-b9a9-4218-a8a7-c439a48e1421 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:47.365037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e1828-b9a9-4218-a8a7-c439a48e1421 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:47.372766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e1828-b9a9-4218-a8a7-c439a48e1421 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:47.374237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e1828-b9a9-4218-a8a7-c439a48e1421 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-422e1828-b9a9-4218-a8a7-c439a48e1421 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:47.385658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:47.386597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:47.387563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 420/839] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:43:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:47.394955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd6226-6956-4bf0-bb7a-e612799d6b99 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:47.397914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:47.508591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd6226-6956-4bf0-bb7a-e612799d6b99 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:47.551130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce157a-92f2-40be-a009-ba298267b6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:47.551748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce157a-92f2-40be-a009-ba298267b6ed None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:47.557368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce157a-92f2-40be-a009-ba298267b6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:47.557368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce157a-92f2-40be-a009-ba298267b6ed None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:47.558079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce157a-92f2-40be-a009-ba298267b6ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:47.558396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce157a-92f2-40be-a009-ba298267b6ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:47.560457 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce157a-92f2-40be-a009-ba298267b6ed None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:47.579821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 420/840] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Sep 15 10:43:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:47.585020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e1828-b9a9-4218-a8a7-c439a48e1421 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:47.612524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:47.692091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e1828-b9a9-4218-a8a7-c439a48e1421 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:47.727201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f8ff0-fafd-4172-82d9-8af359ec5530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:47.727809 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f8ff0-fafd-4172-82d9-8af359ec5530 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:47.728090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f8ff0-fafd-4172-82d9-8af359ec5530 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:47.728420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f8ff0-fafd-4172-82d9-8af359ec5530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:47.801704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f8ff0-fafd-4172-82d9-8af359ec5530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:47.803205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f8ff0-fafd-4172-82d9-8af359ec5530 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:47.834063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:47.911522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 421/841] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:47 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:43:47.911744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01e62c-fd88-403f-a0f9-778ab4267ba5 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:47.912155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01e62c-fd88-403f-a0f9-778ab4267ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:47.912155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01e62c-fd88-403f-a0f9-778ab4267ba5 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:47.913804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 421/842] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:47.916935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f8ff0-fafd-4172-82d9-8af359ec5530 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:47.920244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a46b9-4d2b-4708-8511-60ae553cde72 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:47.920792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a46b9-4d2b-4708-8511-60ae553cde72 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:47.921324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a46b9-4d2b-4708-8511-60ae553cde72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:47.940915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-505a46b9-4d2b-4708-8511-60ae553cde72 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:47.942459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-505a46b9-4d2b-4708-8511-60ae553cde72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:48.002017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:48.021054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 422/843] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:43:48.026557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f8ff0-fafd-4172-82d9-8af359ec5530 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:48.050841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:48.062886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09c9e3-fc55-402f-afea-6a079dd55678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:48.063546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c9e3-fc55-402f-afea-6a079dd55678 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.064168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c9e3-fc55-402f-afea-6a079dd55678 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.064877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c9e3-fc55-402f-afea-6a079dd55678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.078231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09c9e3-fc55-402f-afea-6a079dd55678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:48.079723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09c9e3-fc55-402f-afea-6a079dd55678 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-8e09c9e3-fc55-402f-afea-6a079dd55678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:48.097533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:48.098563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:48.099501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:48.099501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:48.099501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:48.099501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 422/844] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:48.104642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a46b9-4d2b-4708-8511-60ae553cde72 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:48.195635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6617c4b-4fc4-4acc-bd3f-550499828aeb None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.195999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6617c4b-4fc4-4acc-bd3f-550499828aeb None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.196186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6617c4b-4fc4-4acc-bd3f-550499828aeb None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.197816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 423/845] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:43:48.205414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00ae13-5e7a-40c2-a640-256ee1005f85 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.205958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00ae13-5e7a-40c2-a640-256ee1005f85 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.206449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00ae13-5e7a-40c2-a640-256ee1005f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.214595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a46b9-4d2b-4708-8511-60ae553cde72 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:48.226657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4f00ae13-5e7a-40c2-a640-256ee1005f85 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:48.228439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4f00ae13-5e7a-40c2-a640-256ee1005f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:48.258207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b0cce-656b-424f-b3ad-55229ab01bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:48.259785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b0cce-656b-424f-b3ad-55229ab01bce None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.260305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b0cce-656b-424f-b3ad-55229ab01bce None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.261025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b0cce-656b-424f-b3ad-55229ab01bce None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.262105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b0cce-656b-424f-b3ad-55229ab01bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:48.263652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b0cce-656b-424f-b3ad-55229ab01bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:48.265920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b0cce-656b-424f-b3ad-55229ab01bce None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:48.281834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 423/846] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:48.287473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b0cce-656b-424f-b3ad-55229ab01bce None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:48.309278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 424/847] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:48.310344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00ae13-5e7a-40c2-a640-256ee1005f85 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:48.350770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:48.417316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b0cce-656b-424f-b3ad-55229ab01bce None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:48.430961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00ae13-5e7a-40c2-a640-256ee1005f85 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:48.440245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:48.462019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a075ae3-dc5a-45e4-bd95-dfeb05e8033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:48.462944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a075ae3-dc5a-45e4-bd95-dfeb05e8033a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.463590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a075ae3-dc5a-45e4-bd95-dfeb05e8033a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.464019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a075ae3-dc5a-45e4-bd95-dfeb05e8033a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.473718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a075ae3-dc5a-45e4-bd95-dfeb05e8033a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:48.474077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59e43c-777f-4b4c-9a13-31bda917527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:48.474827 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59e43c-777f-4b4c-9a13-31bda917527f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.475260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59e43c-777f-4b4c-9a13-31bda917527f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.475647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59e43c-777f-4b4c-9a13-31bda917527f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.476199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a075ae3-dc5a-45e4-bd95-dfeb05e8033a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:48.476444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59e43c-777f-4b4c-9a13-31bda917527f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:48.478194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59e43c-777f-4b4c-9a13-31bda917527f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-6d59e43c-777f-4b4c-9a13-31bda917527f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:48.489637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:48.490791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:48.491664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 425/848] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5a075ae3-dc5a-45e4-bd95-dfeb05e8033a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:48.494543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:48.495600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:48.496520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:48.496520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:48.496520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:48.496520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:48.496520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:48.496520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:48.496520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:48.496520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:48.499243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59e43c-777f-4b4c-9a13-31bda917527f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:48.501651 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 424/849] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:48.502821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a075ae3-dc5a-45e4-bd95-dfeb05e8033a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:48.624066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59e43c-777f-4b4c-9a13-31bda917527f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:48.629091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a075ae3-dc5a-45e4-bd95-dfeb05e8033a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:48.639174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:48.659043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddf2be-2f20-454a-8f53-f0180f391120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:48.659652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddf2be-2f20-454a-8f53-f0180f391120 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.659924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddf2be-2f20-454a-8f53-f0180f391120 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.660188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddf2be-2f20-454a-8f53-f0180f391120 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.660823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baddf2be-2f20-454a-8f53-f0180f391120 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:48.661134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baddf2be-2f20-454a-8f53-f0180f391120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:48.662509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baddf2be-2f20-454a-8f53-f0180f391120 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:48.663617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d98e7e-1bb2-4da8-9d17-6555e145814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:48.664223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d98e7e-1bb2-4da8-9d17-6555e145814b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.664503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d98e7e-1bb2-4da8-9d17-6555e145814b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.664776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d98e7e-1bb2-4da8-9d17-6555e145814b None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.665492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d98e7e-1bb2-4da8-9d17-6555e145814b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:48.665794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d98e7e-1bb2-4da8-9d17-6555e145814b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:48.666765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 426/850] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:48.667222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d98e7e-1bb2-4da8-9d17-6555e145814b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:48.667882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f7cc8-bed8-4904-8474-d8bb674a27a0 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.668162 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f7cc8-bed8-4904-8474-d8bb674a27a0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.668432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f7cc8-bed8-4904-8474-d8bb674a27a0 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.670167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 427/851] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:48.670704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f7cc8-bed8-4904-8474-d8bb674a27a0 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:48.684237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 425/852] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:48.685113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d98e7e-1bb2-4da8-9d17-6555e145814b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:48.707076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:48.774636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f7cc8-bed8-4904-8474-d8bb674a27a0 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:48.782950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d98e7e-1bb2-4da8-9d17-6555e145814b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:48.808448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef3509-5470-465f-9558-e29766a44a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:48.809106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef3509-5470-465f-9558-e29766a44a0d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.809388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef3509-5470-465f-9558-e29766a44a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.809666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef3509-5470-465f-9558-e29766a44a0d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.810746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef3509-5470-465f-9558-e29766a44a0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:48.812122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef3509-5470-465f-9558-e29766a44a0d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:48.816748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b486a1-a43f-47a5-b0c6-6e9e3a4eee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:48.817372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b486a1-a43f-47a5-b0c6-6e9e3a4eee09 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.817650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b486a1-a43f-47a5-b0c6-6e9e3a4eee09 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.817918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b486a1-a43f-47a5-b0c6-6e9e3a4eee09 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.818528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b486a1-a43f-47a5-b0c6-6e9e3a4eee09 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:48.818823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b486a1-a43f-47a5-b0c6-6e9e3a4eee09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:48.820164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b486a1-a43f-47a5-b0c6-6e9e3a4eee09 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-37ef3509-5470-465f-9558-e29766a44a0d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:48.820939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:48.821957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:48.822919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 428/853] 10.222.0.50 () {56 vars in 1132 bytes} [Thu Sep 15 10:43:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:48.824439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010d46c-d071-4b2c-af2d-861623dcb5f6 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:48.824439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010d46c-d071-4b2c-af2d-861623dcb5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:48.824439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010d46c-d071-4b2c-af2d-861623dcb5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:48.828197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 426/854] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:48.829194 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b486a1-a43f-47a5-b0c6-6e9e3a4eee09 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:48.847572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-a010d46c-d071-4b2c-af2d-861623dcb5f6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:48.848842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a010d46c-d071-4b2c-af2d-861623dcb5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:48.889177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:48.925345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 429/855] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:48.926222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010d46c-d071-4b2c-af2d-861623dcb5f6 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:48.967639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:48.987734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b486a1-a43f-47a5-b0c6-6e9e3a4eee09 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:49.028310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176911-f207-497c-8010-73602c089377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:49.029090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176911-f207-497c-8010-73602c089377 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:49.029416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176911-f207-497c-8010-73602c089377 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:49.029745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176911-f207-497c-8010-73602c089377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:49.030466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176911-f207-497c-8010-73602c089377 None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=06007b3b2260409f863ac3d0ac85f4cc) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:49.030768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176911-f207-497c-8010-73602c089377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:49.032199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176911-f207-497c-8010-73602c089377 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:49.038414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 427/856] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:48 2022] GET /identity/v3/role_assignments?user.id=06007b3b2260409f863ac3d0ac85f4cc&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=0be9777002c641b4a47f78df647cc173 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:49.062802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010d46c-d071-4b2c-af2d-861623dcb5f6 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:49.104253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adb68c-ca2c-4644-bd89-00484e3a9e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:49.104916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adb68c-ca2c-4644-bd89-00484e3a9e87 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:49.105201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adb68c-ca2c-4644-bd89-00484e3a9e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:49.105475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adb68c-ca2c-4644-bd89-00484e3a9e87 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:49.106126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adb68c-ca2c-4644-bd89-00484e3a9e87 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:49.106431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adb68c-ca2c-4644-bd89-00484e3a9e87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:49.107879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adb68c-ca2c-4644-bd89-00484e3a9e87 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:49.113619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 430/857] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:43:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:49.119641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176911-f207-497c-8010-73602c089377 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:49.225278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adb68c-ca2c-4644-bd89-00484e3a9e87 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:49.244039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:49.248727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176911-f207-497c-8010-73602c089377 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:49.293902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa468e67-89f6-475b-901c-2bc766ddf0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:49.295826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa468e67-89f6-475b-901c-2bc766ddf0ad None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:49.296571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa468e67-89f6-475b-901c-2bc766ddf0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:49.297524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa468e67-89f6-475b-901c-2bc766ddf0ad None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:49.298643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa468e67-89f6-475b-901c-2bc766ddf0ad None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:49.299423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa468e67-89f6-475b-901c-2bc766ddf0ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:49.301761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa468e67-89f6-475b-901c-2bc766ddf0ad None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:49.308447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 428/858] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:43:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:49.339946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:49.351998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adb68c-ca2c-4644-bd89-00484e3a9e87 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:49.394964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1500ed-c92f-4e58-8984-f2475a371812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:49.395626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1500ed-c92f-4e58-8984-f2475a371812 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:49.395903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1500ed-c92f-4e58-8984-f2475a371812 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:49.396171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1500ed-c92f-4e58-8984-f2475a371812 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:49.409284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1500ed-c92f-4e58-8984-f2475a371812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-7f1500ed-c92f-4e58-8984-f2475a371812 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:49.414919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:49.415926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:49.417066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 431/859] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:43:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:49.425614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa468e67-89f6-475b-901c-2bc766ddf0ad None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:49.543579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa468e67-89f6-475b-901c-2bc766ddf0ad None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:49.582516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd5d64-4c77-4b09-8c95-ab7cc3460271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:49.583281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd5d64-4c77-4b09-8c95-ab7cc3460271 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:49.583665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd5d64-4c77-4b09-8c95-ab7cc3460271 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:49.584043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd5d64-4c77-4b09-8c95-ab7cc3460271 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:49.584827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd5d64-4c77-4b09-8c95-ab7cc3460271 None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:49.585273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd5d64-4c77-4b09-8c95-ab7cc3460271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:49.586812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd5d64-4c77-4b09-8c95-ab7cc3460271 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:49.600677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 429/860] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:43:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:49.602306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1500ed-c92f-4e58-8984-f2475a371812 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:49.716183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1500ed-c92f-4e58-8984-f2475a371812 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:49.752079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57477695-ef8a-4060-833b-8a9c0ac9bf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:49.752711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57477695-ef8a-4060-833b-8a9c0ac9bf83 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:49.753021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57477695-ef8a-4060-833b-8a9c0ac9bf83 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:49.753291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57477695-ef8a-4060-833b-8a9c0ac9bf83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:49.765511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57477695-ef8a-4060-833b-8a9c0ac9bf83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:49.766974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57477695-ef8a-4060-833b-8a9c0ac9bf83 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-57477695-ef8a-4060-833b-8a9c0ac9bf83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:49.781947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:49.782947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:49.783866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:49.783866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:49.783866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:49.783866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 432/861] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:49.788837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd5d64-4c77-4b09-8c95-ab7cc3460271 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:49.817484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:49.903299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd5d64-4c77-4b09-8c95-ab7cc3460271 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:49.939570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d30c72-93e5-47f6-8f66-8eeefc988ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:49.940182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d30c72-93e5-47f6-8f66-8eeefc988ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:49.940461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d30c72-93e5-47f6-8f66-8eeefc988ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:49.940735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d30c72-93e5-47f6-8f66-8eeefc988ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:49.941426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d30c72-93e5-47f6-8f66-8eeefc988ec1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:49.941732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d30c72-93e5-47f6-8f66-8eeefc988ec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:49.943114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d30c72-93e5-47f6-8f66-8eeefc988ec1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:49.961800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 430/862] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:49.963130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57477695-ef8a-4060-833b-8a9c0ac9bf83 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:49.968653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:50.079780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57477695-ef8a-4060-833b-8a9c0ac9bf83 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:50.115664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67550c6-3d1d-43e8-989d-f5d37909fc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:50.116294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67550c6-3d1d-43e8-989d-f5d37909fc40 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:50.116621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67550c6-3d1d-43e8-989d-f5d37909fc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:50.116923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67550c6-3d1d-43e8-989d-f5d37909fc40 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:50.125234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67550c6-3d1d-43e8-989d-f5d37909fc40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:50.126691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67550c6-3d1d-43e8-989d-f5d37909fc40 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-f67550c6-3d1d-43e8-989d-f5d37909fc40 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:50.138742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:50.140648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:50.141703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 433/863] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:43:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:50.145737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d30c72-93e5-47f6-8f66-8eeefc988ec1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:50.240787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:50.261380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d30c72-93e5-47f6-8f66-8eeefc988ec1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:50.301740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698e86c-9224-4b88-a031-d6989fe918bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:50.302353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698e86c-9224-4b88-a031-d6989fe918bf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:50.302633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698e86c-9224-4b88-a031-d6989fe918bf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:50.302905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698e86c-9224-4b88-a031-d6989fe918bf None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:50.303624 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698e86c-9224-4b88-a031-d6989fe918bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:50.303928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698e86c-9224-4b88-a031-d6989fe918bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:50.305393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698e86c-9224-4b88-a031-d6989fe918bf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:50.323745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 431/864] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Sep 15 10:43:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:50.330298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67550c6-3d1d-43e8-989d-f5d37909fc40 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:50.394891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:50.437169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67550c6-3d1d-43e8-989d-f5d37909fc40 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:50.471764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8c0fd-0cf4-425a-84c0-c54006b9d671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:50.472364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8c0fd-0cf4-425a-84c0-c54006b9d671 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:50.472642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8c0fd-0cf4-425a-84c0-c54006b9d671 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:50.472956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8c0fd-0cf4-425a-84c0-c54006b9d671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:50.542100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8c0fd-0cf4-425a-84c0-c54006b9d671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:50.548834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8c0fd-0cf4-425a-84c0-c54006b9d671 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:50.606689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:50.646752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 434/865] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:43:50.652238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698e86c-9224-4b88-a031-d6989fe918bf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:50.764702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698e86c-9224-4b88-a031-d6989fe918bf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:50.800972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6fde5-e673-44dc-9638-08268bcd0494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:50.801610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6fde5-e673-44dc-9638-08268bcd0494 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:50.801892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6fde5-e673-44dc-9638-08268bcd0494 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:50.802166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6fde5-e673-44dc-9638-08268bcd0494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:50.813222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6fde5-e673-44dc-9638-08268bcd0494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:50.814750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6fde5-e673-44dc-9638-08268bcd0494 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:50.816434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c4c6fde5-e673-44dc-9638-08268bcd0494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:50.830679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:50.831722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:50.832640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:50.832640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:50.832640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:50.832640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 432/866] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:50.838003 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8c0fd-0cf4-425a-84c0-c54006b9d671 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:50.945930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8c0fd-0cf4-425a-84c0-c54006b9d671 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:50.982271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae28113-2ac7-4423-b22b-ca0539d1ed97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:50.983154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae28113-2ac7-4423-b22b-ca0539d1ed97 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:50.983434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae28113-2ac7-4423-b22b-ca0539d1ed97 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:50.983700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae28113-2ac7-4423-b22b-ca0539d1ed97 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:50.984344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae28113-2ac7-4423-b22b-ca0539d1ed97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:50.984642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae28113-2ac7-4423-b22b-ca0539d1ed97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:50.986050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae28113-2ac7-4423-b22b-ca0539d1ed97 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:50.998521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 435/867] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:51.003801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6fde5-e673-44dc-9638-08268bcd0494 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:51.113868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6fde5-e673-44dc-9638-08268bcd0494 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:51.155114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1bb87f-18ca-44ea-87a4-c6000eda033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:51.155906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bb87f-18ca-44ea-87a4-c6000eda033b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.156095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bb87f-18ca-44ea-87a4-c6000eda033b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.156382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bb87f-18ca-44ea-87a4-c6000eda033b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.163637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1bb87f-18ca-44ea-87a4-c6000eda033b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:51.165199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1bb87f-18ca-44ea-87a4-c6000eda033b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2e1bb87f-18ca-44ea-87a4-c6000eda033b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:51.177093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:51.179157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:51.181352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:51.181352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:51.181352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:51.181352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:51.181352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:51.181352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:51.181352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:51.181352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:51.185376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 433/868] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:51.186384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae28113-2ac7-4423-b22b-ca0539d1ed97 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:51.215556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:51.238496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e8287-dabb-49dd-b037-54957cf3821c None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.238792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e8287-dabb-49dd-b037-54957cf3821c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.239076 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e8287-dabb-49dd-b037-54957cf3821c None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.240831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 434/869] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:43:51.248017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b032ce7-c3b1-4500-a503-06d091324069 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.248307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b032ce7-c3b1-4500-a503-06d091324069 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.248587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b032ce7-c3b1-4500-a503-06d091324069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.273652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2b032ce7-c3b1-4500-a503-06d091324069 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:51.275602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2b032ce7-c3b1-4500-a503-06d091324069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:51.297788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae28113-2ac7-4423-b22b-ca0539d1ed97 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:51.339691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d45854-4994-4379-a2f9-fb8a80621766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:51.340629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d45854-4994-4379-a2f9-fb8a80621766 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.341825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d45854-4994-4379-a2f9-fb8a80621766 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.342431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d45854-4994-4379-a2f9-fb8a80621766 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.343324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d45854-4994-4379-a2f9-fb8a80621766 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:51.343804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d45854-4994-4379-a2f9-fb8a80621766 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:51.345595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d45854-4994-4379-a2f9-fb8a80621766 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:51.359185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 436/870] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:51.359503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:51.366187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d45854-4994-4379-a2f9-fb8a80621766 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:51.372889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 435/871] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:51.431735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130eca4-d301-47e8-8b4a-488b053c9079 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.432057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130eca4-d301-47e8-8b4a-488b053c9079 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.432301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130eca4-d301-47e8-8b4a-488b053c9079 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.433805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 436/872] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:43:51.439697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c81e3-21ea-4112-a3e9-7f2e8d508763 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.439986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c81e3-21ea-4112-a3e9-7f2e8d508763 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.440266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c81e3-21ea-4112-a3e9-7f2e8d508763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.459481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3a2c81e3-21ea-4112-a3e9-7f2e8d508763 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:51.460685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3a2c81e3-21ea-4112-a3e9-7f2e8d508763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:51.473974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:51.494576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d45854-4994-4379-a2f9-fb8a80621766 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:51.533406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85c646-0e6d-4e93-8821-56d174cc6954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:51.533995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85c646-0e6d-4e93-8821-56d174cc6954 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.534276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85c646-0e6d-4e93-8821-56d174cc6954 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.534550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85c646-0e6d-4e93-8821-56d174cc6954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.535274 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85c646-0e6d-4e93-8821-56d174cc6954 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=0991767dd90d4f62bfe5484aad435669) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:51.535599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85c646-0e6d-4e93-8821-56d174cc6954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:51.537070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85c646-0e6d-4e93-8821-56d174cc6954 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:51.548988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 437/873] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:51 2022] GET /identity/v3/role_assignments?user.id=0991767dd90d4f62bfe5484aad435669&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:51.566976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 437/874] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:43:51.604291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603e852-8334-4810-a9dd-481e9388e9ed None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.604575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603e852-8334-4810-a9dd-481e9388e9ed None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.604888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603e852-8334-4810-a9dd-481e9388e9ed None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.606606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 438/875] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:51.612671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c5fab-20f1-44c4-84e2-001c63efced1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.613010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c5fab-20f1-44c4-84e2-001c63efced1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.613289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c5fab-20f1-44c4-84e2-001c63efced1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.633507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-333c5fab-20f1-44c4-84e2-001c63efced1 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:51.634744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-333c5fab-20f1-44c4-84e2-001c63efced1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:51.688303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603e852-8334-4810-a9dd-481e9388e9ed None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:51.724292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 438/876] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:51.822800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603e852-8334-4810-a9dd-481e9388e9ed None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:51.844966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c5fab-20f1-44c4-84e2-001c63efced1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:51.865461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec54c4-bde9-4e1b-86a4-6e14ceb64375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:51.866077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec54c4-bde9-4e1b-86a4-6e14ceb64375 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:51.866359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec54c4-bde9-4e1b-86a4-6e14ceb64375 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:51.866630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec54c4-bde9-4e1b-86a4-6e14ceb64375 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:51.867284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec54c4-bde9-4e1b-86a4-6e14ceb64375 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:51.868662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec54c4-bde9-4e1b-86a4-6e14ceb64375 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:51.889973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 439/877] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:43:51 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:51.908590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:51.975584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c5fab-20f1-44c4-84e2-001c63efced1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:52.014401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec54c4-bde9-4e1b-86a4-6e14ceb64375 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:52.023738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658952ad-e93d-43d1-b0cf-585a3ac1547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:52.024385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658952ad-e93d-43d1-b0cf-585a3ac1547c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:52.024668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658952ad-e93d-43d1-b0cf-585a3ac1547c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:52.024982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658952ad-e93d-43d1-b0cf-585a3ac1547c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:52.025581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658952ad-e93d-43d1-b0cf-585a3ac1547c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:52.027026 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658952ad-e93d-43d1-b0cf-585a3ac1547c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:52.029075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-658952ad-e93d-43d1-b0cf-585a3ac1547c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:52.040136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:52.041292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:52.042743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 439/878] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:43:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:52.047352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658952ad-e93d-43d1-b0cf-585a3ac1547c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:52.154899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec54c4-bde9-4e1b-86a4-6e14ceb64375 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:52.168308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658952ad-e93d-43d1-b0cf-585a3ac1547c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:52.174759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:52.205898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf1fda-f1c1-4862-97b3-606005c7353e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:52.206898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf1fda-f1c1-4862-97b3-606005c7353e None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:52.207339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf1fda-f1c1-4862-97b3-606005c7353e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:52.207801 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf1fda-f1c1-4862-97b3-606005c7353e None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:52.208617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf1fda-f1c1-4862-97b3-606005c7353e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:52.210255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf1fda-f1c1-4862-97b3-606005c7353e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:52.212400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d19015-b664-41df-926d-5c6953838919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:52.213036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d19015-b664-41df-926d-5c6953838919 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:52.213378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d19015-b664-41df-926d-5c6953838919 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:52.213710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d19015-b664-41df-926d-5c6953838919 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:52.214394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d19015-b664-41df-926d-5c6953838919 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:52.214701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d19015-b664-41df-926d-5c6953838919 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:52.216677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d19015-b664-41df-926d-5c6953838919 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:52.222386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 440/879] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:43:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:52.227639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d19015-b664-41df-926d-5c6953838919 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:52.236919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 440/880] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:43:52 2022] POST /identity/v3/services => generated 269 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:43:52.348573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d19015-b664-41df-926d-5c6953838919 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:52.395696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e93aa8-7130-4f39-a85d-7e68eae52840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:52.396566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e93aa8-7130-4f39-a85d-7e68eae52840 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:52.397022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e93aa8-7130-4f39-a85d-7e68eae52840 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:52.397463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e93aa8-7130-4f39-a85d-7e68eae52840 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:52.398236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e93aa8-7130-4f39-a85d-7e68eae52840 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:52.398697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e93aa8-7130-4f39-a85d-7e68eae52840 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:52.400250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e93aa8-7130-4f39-a85d-7e68eae52840 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:52.406759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 441/881] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:43:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:53.944853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce8c13-8175-408c-ad85-e38ac480dd30 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:53.945901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce8c13-8175-408c-ad85-e38ac480dd30 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:53.946423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce8c13-8175-408c-ad85-e38ac480dd30 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:53.948214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 441/882] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:53.954058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b8f73-f77a-4edb-a63a-d7135ec37a07 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:53.954886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b8f73-f77a-4edb-a63a-d7135ec37a07 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:53.955381 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b8f73-f77a-4edb-a63a-d7135ec37a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:53.974807 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-346b8f73-f77a-4edb-a63a-d7135ec37a07 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:53.976344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-346b8f73-f77a-4edb-a63a-d7135ec37a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:54.000956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:54.049188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 442/883] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:54.237192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e8cf7-63c8-44c5-bcdc-5ca63b711666 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.237815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e8cf7-63c8-44c5-bcdc-5ca63b711666 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.238283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e8cf7-63c8-44c5-bcdc-5ca63b711666 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.240089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 442/884] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:54.245487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffdced-ea19-4dc7-bd0e-dbba0316d653 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.246002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffdced-ea19-4dc7-bd0e-dbba0316d653 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.246548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffdced-ea19-4dc7-bd0e-dbba0316d653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.265927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-81ffdced-ea19-4dc7-bd0e-dbba0316d653 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:54.267694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-81ffdced-ea19-4dc7-bd0e-dbba0316d653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:54.327340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e8cf7-63c8-44c5-bcdc-5ca63b711666 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:54.338398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 443/885] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:54.409909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:54.447301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e8cf7-63c8-44c5-bcdc-5ca63b711666 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:54.483904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f637bc-e8b9-44b7-b655-ccb883f4fa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:54.484565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f637bc-e8b9-44b7-b655-ccb883f4fa1f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.484866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f637bc-e8b9-44b7-b655-ccb883f4fa1f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.485139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f637bc-e8b9-44b7-b655-ccb883f4fa1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.498625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f637bc-e8b9-44b7-b655-ccb883f4fa1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-42f637bc-e8b9-44b7-b655-ccb883f4fa1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:43:54.504242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:54.505315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:54.506200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 443/886] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Sep 15 10:43:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:54.513105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06164a-14f6-44a6-aff2-81cf0161ac05 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.513710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06164a-14f6-44a6-aff2-81cf0161ac05 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.514190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06164a-14f6-44a6-aff2-81cf0161ac05 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.516053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f637bc-e8b9-44b7-b655-ccb883f4fa1f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:54.516547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 444/887] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:54.521401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5e885-000b-4685-a488-b40bd7a25a48 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.521915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5e885-000b-4685-a488-b40bd7a25a48 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.522390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5e885-000b-4685-a488-b40bd7a25a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.546695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4af5e885-000b-4685-a488-b40bd7a25a48 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:54.548233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4af5e885-000b-4685-a488-b40bd7a25a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:54.636131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f637bc-e8b9-44b7-b655-ccb883f4fa1f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:54.638397 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 445/888] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:54.639559 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c80ae-692c-4088-9560-6c6fa5871789 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.639850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c80ae-692c-4088-9560-6c6fa5871789 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.640123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c80ae-692c-4088-9560-6c6fa5871789 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.641594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 446/889] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:54.642119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c80ae-692c-4088-9560-6c6fa5871789 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:54.661205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:54.672290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:54.682309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561ad06-4616-4a20-82ff-282216819e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:54.683130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561ad06-4616-4a20-82ff-282216819e71 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.683584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561ad06-4616-4a20-82ff-282216819e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.684048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561ad06-4616-4a20-82ff-282216819e71 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.685795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561ad06-4616-4a20-82ff-282216819e71 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:54.686280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561ad06-4616-4a20-82ff-282216819e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:54.688493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561ad06-4616-4a20-82ff-282216819e71 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:54.698971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 444/890] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Sep 15 10:43:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:54.700105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb743de9-c832-4179-9a24-8e00e93f414a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.700380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb743de9-c832-4179-9a24-8e00e93f414a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.700672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb743de9-c832-4179-9a24-8e00e93f414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.720380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-bb743de9-c832-4179-9a24-8e00e93f414a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:54.721610 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-bb743de9-c832-4179-9a24-8e00e93f414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:54.752199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c80ae-692c-4088-9560-6c6fa5871789 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:54.788160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 445/891] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:54.788818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb743de9-c832-4179-9a24-8e00e93f414a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:54.793131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef80f3c-00ca-4c18-971d-b558da88fe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:54.794640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef80f3c-00ca-4c18-971d-b558da88fe87 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.795137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef80f3c-00ca-4c18-971d-b558da88fe87 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.795637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef80f3c-00ca-4c18-971d-b558da88fe87 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.796431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef80f3c-00ca-4c18-971d-b558da88fe87 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:54.798044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef80f3c-00ca-4c18-971d-b558da88fe87 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-7ef80f3c-00ca-4c18-971d-b558da88fe87 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:54.810412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:54.811469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 10:43:54.812340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:54.816773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 447/892] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:54.818232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef80f3c-00ca-4c18-971d-b558da88fe87 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:54.895865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb743de9-c832-4179-9a24-8e00e93f414a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:54.916963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:54.925639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:54.934025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c6fb4-15ab-4f9b-8ddf-ae2eb0ed371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:54.935309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c6fb4-15ab-4f9b-8ddf-ae2eb0ed371e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.935591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c6fb4-15ab-4f9b-8ddf-ae2eb0ed371e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.935856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c6fb4-15ab-4f9b-8ddf-ae2eb0ed371e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.945870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c6fb4-15ab-4f9b-8ddf-ae2eb0ed371e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:54.947310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c6fb4-15ab-4f9b-8ddf-ae2eb0ed371e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:54.949393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef80f3c-00ca-4c18-971d-b558da88fe87 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-169c6fb4-15ab-4f9b-8ddf-ae2eb0ed371e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:54.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:54.964967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:54.965984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:54.965984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:54.965984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:54.965984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 446/893] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:54.966233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c6fb4-15ab-4f9b-8ddf-ae2eb0ed371e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:54.992268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064692a-d38b-4efc-9bb8-ef5d03e1c947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:54.993486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064692a-d38b-4efc-9bb8-ef5d03e1c947 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:54.993763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064692a-d38b-4efc-9bb8-ef5d03e1c947 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:54.994032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064692a-d38b-4efc-9bb8-ef5d03e1c947 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:54.994647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064692a-d38b-4efc-9bb8-ef5d03e1c947 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:54.994946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064692a-d38b-4efc-9bb8-ef5d03e1c947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:54.996338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064692a-d38b-4efc-9bb8-ef5d03e1c947 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.004167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 448/894] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:55.004996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064692a-d38b-4efc-9bb8-ef5d03e1c947 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.078326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c6fb4-15ab-4f9b-8ddf-ae2eb0ed371e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.108098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064692a-d38b-4efc-9bb8-ef5d03e1c947 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.111682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dbc0d-da9b-40cc-9774-170e710baa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.112307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dbc0d-da9b-40cc-9774-170e710baa1e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.112585 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dbc0d-da9b-40cc-9774-170e710baa1e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.112887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dbc0d-da9b-40cc-9774-170e710baa1e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.113465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54dbc0d-da9b-40cc-9774-170e710baa1e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.114836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54dbc0d-da9b-40cc-9774-170e710baa1e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-a54dbc0d-da9b-40cc-9774-170e710baa1e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:55.124354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:55.125528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:55.126648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 447/895] 10.222.0.50 () {56 vars in 1132 bytes} [Thu Sep 15 10:43:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:55.128593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dbc0d-da9b-40cc-9774-170e710baa1e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.141815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758b141-2c63-45ba-9048-b487c5965ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.142424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758b141-2c63-45ba-9048-b487c5965ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.142700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758b141-2c63-45ba-9048-b487c5965ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.142968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758b141-2c63-45ba-9048-b487c5965ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.143595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758b141-2c63-45ba-9048-b487c5965ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:55.143897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758b141-2c63-45ba-9048-b487c5965ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.145382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758b141-2c63-45ba-9048-b487c5965ab9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.157736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 449/896] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:55.158526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758b141-2c63-45ba-9048-b487c5965ab9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.166817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:55.167014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:55.233241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dbc0d-da9b-40cc-9774-170e710baa1e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.257329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758b141-2c63-45ba-9048-b487c5965ab9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.267509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bca76-fd3d-492f-a012-9f5f5747977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.268112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bca76-fd3d-492f-a012-9f5f5747977b None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.268388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bca76-fd3d-492f-a012-9f5f5747977b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.268656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bca76-fd3d-492f-a012-9f5f5747977b None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.269304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bca76-fd3d-492f-a012-9f5f5747977b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:55.269602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bca76-fd3d-492f-a012-9f5f5747977b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.270964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bca76-fd3d-492f-a012-9f5f5747977b None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.275822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 448/897] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:55.276420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bca76-fd3d-492f-a012-9f5f5747977b None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.294753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b853d-463d-450b-85be-8b62ede8e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.295357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b853d-463d-450b-85be-8b62ede8e127 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.295632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b853d-463d-450b-85be-8b62ede8e127 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.295903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b853d-463d-450b-85be-8b62ede8e127 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.296533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b853d-463d-450b-85be-8b62ede8e127 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.297947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b853d-463d-450b-85be-8b62ede8e127 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.312066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 450/898] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:43:55 2022] POST /identity/v3/services => generated 241 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:55.312877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b853d-463d-450b-85be-8b62ede8e127 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.393322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:55.395727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:55.397247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bca76-fd3d-492f-a012-9f5f5747977b None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.442479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b853d-463d-450b-85be-8b62ede8e127 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.444618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8568f-f48d-4953-8795-ddaa56841ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.445288 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8568f-f48d-4953-8795-ddaa56841ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.445571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8568f-f48d-4953-8795-ddaa56841ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.445850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8568f-f48d-4953-8795-ddaa56841ff6 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.446488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8568f-f48d-4953-8795-ddaa56841ff6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:55.446794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8568f-f48d-4953-8795-ddaa56841ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.448191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8568f-f48d-4953-8795-ddaa56841ff6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.460185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 449/899] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:55.460185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8568f-f48d-4953-8795-ddaa56841ff6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.489813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461869a-3071-49f5-9f6e-c756fca8d0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.490714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461869a-3071-49f5-9f6e-c756fca8d0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.491437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461869a-3071-49f5-9f6e-c756fca8d0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.491718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461869a-3071-49f5-9f6e-c756fca8d0f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.503184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461869a-3071-49f5-9f6e-c756fca8d0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.505118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461869a-3071-49f5-9f6e-c756fca8d0f5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e461869a-3071-49f5-9f6e-c756fca8d0f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:43:55.518625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:55.519602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 451/900] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:55.521364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461869a-3071-49f5-9f6e-c756fca8d0f5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.585400 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8568f-f48d-4953-8795-ddaa56841ff6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.631546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a39c42-9b02-4022-81fa-5ebf50f8f5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.632238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a39c42-9b02-4022-81fa-5ebf50f8f5fb None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.632494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a39c42-9b02-4022-81fa-5ebf50f8f5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.632772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a39c42-9b02-4022-81fa-5ebf50f8f5fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.633556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a39c42-9b02-4022-81fa-5ebf50f8f5fb None admin] RBAC: Adding query filter params (interface=public, service_id=cc4c186217564652a04f2332a2158ecc, region_id=RegionOne) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:55.633807 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a39c42-9b02-4022-81fa-5ebf50f8f5fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.635182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a39c42-9b02-4022-81fa-5ebf50f8f5fb None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.635915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461869a-3071-49f5-9f6e-c756fca8d0f5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.641098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 450/901] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/endpoints?service_id=cc4c186217564652a04f2332a2158ecc&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:55.641901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a39c42-9b02-4022-81fa-5ebf50f8f5fb None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.664770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:55.672192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:55.678736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83894435-9fbf-4912-bea3-38550890488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.679381 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83894435-9fbf-4912-bea3-38550890488f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.679633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83894435-9fbf-4912-bea3-38550890488f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.679905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83894435-9fbf-4912-bea3-38550890488f None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.680527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83894435-9fbf-4912-bea3-38550890488f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:55.680857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83894435-9fbf-4912-bea3-38550890488f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.682235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83894435-9fbf-4912-bea3-38550890488f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.692427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 452/902] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:55.692427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83894435-9fbf-4912-bea3-38550890488f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.756601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a39c42-9b02-4022-81fa-5ebf50f8f5fb None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.793015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7e571-d7ec-4d40-9d3a-18aca345f8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.794102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7e571-d7ec-4d40-9d3a-18aca345f8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.794665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7e571-d7ec-4d40-9d3a-18aca345f8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.794966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7e571-d7ec-4d40-9d3a-18aca345f8e4 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.795798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7e571-d7ec-4d40-9d3a-18aca345f8e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:55.796106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7e571-d7ec-4d40-9d3a-18aca345f8e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.797696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7e571-d7ec-4d40-9d3a-18aca345f8e4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.800967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83894435-9fbf-4912-bea3-38550890488f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.822630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 451/903] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:55.823821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7e571-d7ec-4d40-9d3a-18aca345f8e4 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.840476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b7127-70e9-48bb-9de3-85f248b779e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:55.841300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b7127-70e9-48bb-9de3-85f248b779e8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:55.841586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b7127-70e9-48bb-9de3-85f248b779e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:55.841862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b7127-70e9-48bb-9de3-85f248b779e8 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:55.842461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b7127-70e9-48bb-9de3-85f248b779e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:55.843850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b7127-70e9-48bb-9de3-85f248b779e8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:55.850153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 453/904] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:43:55.850776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b7127-70e9-48bb-9de3-85f248b779e8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:55.911258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:55.952870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:55.971052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b7127-70e9-48bb-9de3-85f248b779e8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:55.982533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7e571-d7ec-4d40-9d3a-18aca345f8e4 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:56.025084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf571f-e2cd-49cd-a64a-0e323693864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:56.026454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf571f-e2cd-49cd-a64a-0e323693864b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:56.026798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e93229-aa3a-4e87-8c42-9303ec48f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:56.027104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf571f-e2cd-49cd-a64a-0e323693864b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:56.027382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf571f-e2cd-49cd-a64a-0e323693864b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:56.027493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93229-aa3a-4e87-8c42-9303ec48f02b None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:56.027675 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93229-aa3a-4e87-8c42-9303ec48f02b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:56.027990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93229-aa3a-4e87-8c42-9303ec48f02b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:56.028821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e93229-aa3a-4e87-8c42-9303ec48f02b None admin] RBAC: Adding query filter params (interface=public, service_id=1b7de9906c3848fd9295c195a7174fb8, region_id=RegionOne) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:56.029351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e93229-aa3a-4e87-8c42-9303ec48f02b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:56.030502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e93229-aa3a-4e87-8c42-9303ec48f02b None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:56.038523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 452/905] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/endpoints?service_id=1b7de9906c3848fd9295c195a7174fb8&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:56.044437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e93229-aa3a-4e87-8c42-9303ec48f02b None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:56.112993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf571f-e2cd-49cd-a64a-0e323693864b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:56.115168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf571f-e2cd-49cd-a64a-0e323693864b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:56.165481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e93229-aa3a-4e87-8c42-9303ec48f02b None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:56.181072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:56.202525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0caf34-3d0e-406c-89f6-a0464138b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:56.203236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0caf34-3d0e-406c-89f6-a0464138b0ed None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:56.203576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0caf34-3d0e-406c-89f6-a0464138b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:56.203929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0caf34-3d0e-406c-89f6-a0464138b0ed None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:56.204606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0caf34-3d0e-406c-89f6-a0464138b0ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:56.206108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0caf34-3d0e-406c-89f6-a0464138b0ed None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:56.212910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 453/906] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Sep 15 10:43:56 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:43:56.260311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 454/907] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:43:55 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:43:56.262066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0caf34-3d0e-406c-89f6-a0464138b0ed None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:56.277247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:56.378395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0caf34-3d0e-406c-89f6-a0464138b0ed None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:56.415590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0931f6-c7be-46bb-a1f0-2038c7684f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:56.416219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0931f6-c7be-46bb-a1f0-2038c7684f9e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:56.416492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0931f6-c7be-46bb-a1f0-2038c7684f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:56.416757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0931f6-c7be-46bb-a1f0-2038c7684f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:56.427536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0931f6-c7be-46bb-a1f0-2038c7684f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:56.429013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0931f6-c7be-46bb-a1f0-2038c7684f9e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-3a0931f6-c7be-46bb-a1f0-2038c7684f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:43:56.445180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:56.446184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:43:56.447143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:43:56.447143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:43:56.447143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:56.448882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 454/908] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:56.453511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf571f-e2cd-49cd-a64a-0e323693864b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:56.562506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf571f-e2cd-49cd-a64a-0e323693864b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:56.599440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd72e7-5d5a-463f-ae35-3f3a4554a1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:56.600058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd72e7-5d5a-463f-ae35-3f3a4554a1bc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:56.600336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd72e7-5d5a-463f-ae35-3f3a4554a1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:56.600604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd72e7-5d5a-463f-ae35-3f3a4554a1bc None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:56.601278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd72e7-5d5a-463f-ae35-3f3a4554a1bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:56.601577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd72e7-5d5a-463f-ae35-3f3a4554a1bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:56.603036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd72e7-5d5a-463f-ae35-3f3a4554a1bc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:56.615880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 455/909] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:56.622088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0931f6-c7be-46bb-a1f0-2038c7684f9e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:56.688511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:56.735526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0931f6-c7be-46bb-a1f0-2038c7684f9e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:56.770736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae093b-7a9f-48d2-9057-4751c4b0ab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:56.771645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae093b-7a9f-48d2-9057-4751c4b0ab9d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:56.772080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae093b-7a9f-48d2-9057-4751c4b0ab9d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:56.772502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae093b-7a9f-48d2-9057-4751c4b0ab9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:56.780525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae093b-7a9f-48d2-9057-4751c4b0ab9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:56.782215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae093b-7a9f-48d2-9057-4751c4b0ab9d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-36ae093b-7a9f-48d2-9057-4751c4b0ab9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:43:56.795690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:56.796921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:56.798062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:43:56.798062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:43:56.798062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:43:56.798062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:43:56.798062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:43:56.798062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:43:56.798062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:43:56.798062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:56.800364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 455/910] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:56.806232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd72e7-5d5a-463f-ae35-3f3a4554a1bc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:56.820927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:56.915621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd72e7-5d5a-463f-ae35-3f3a4554a1bc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:56.951923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7c48f-f8ed-4b09-9536-0f0b8d191975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:56.952454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7c48f-f8ed-4b09-9536-0f0b8d191975 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:56.952733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7c48f-f8ed-4b09-9536-0f0b8d191975 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:56.953032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7c48f-f8ed-4b09-9536-0f0b8d191975 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:56.953712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7c48f-f8ed-4b09-9536-0f0b8d191975 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:56.954018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7c48f-f8ed-4b09-9536-0f0b8d191975 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:56.955508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7c48f-f8ed-4b09-9536-0f0b8d191975 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:56.973045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 456/911] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:56.975113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae093b-7a9f-48d2-9057-4751c4b0ab9d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:57.084981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae093b-7a9f-48d2-9057-4751c4b0ab9d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:57.109881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:57.122342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dbe50-6936-409c-967d-349fea80fad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:57.123196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dbe50-6936-409c-967d-349fea80fad2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:57.123637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dbe50-6936-409c-967d-349fea80fad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:57.124224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dbe50-6936-409c-967d-349fea80fad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:57.125274 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dbe50-6936-409c-967d-349fea80fad2 None admin] RBAC: Adding query filter params (role.id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=0991767dd90d4f62bfe5484aad435669) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:57.125876 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dbe50-6936-409c-967d-349fea80fad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:57.127459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dbe50-6936-409c-967d-349fea80fad2 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:57.133923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 456/912] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:43:56 2022] GET /identity/v3/role_assignments?user.id=0991767dd90d4f62bfe5484aad435669&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:57.622618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22745c72-beb4-420b-bc54-dc734f640e5e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:57.623134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22745c72-beb4-420b-bc54-dc734f640e5e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:57.623370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22745c72-beb4-420b-bc54-dc734f640e5e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:57.624986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 457/913] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:57.630502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc844d-324e-4c4d-8dc6-d032de182172 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:57.630791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc844d-324e-4c4d-8dc6-d032de182172 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:57.631077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc844d-324e-4c4d-8dc6-d032de182172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:57.650126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9adc844d-324e-4c4d-8dc6-d032de182172 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:57.651326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9adc844d-324e-4c4d-8dc6-d032de182172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:57.723139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 457/914] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:57.999515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22745c72-beb4-420b-bc54-dc734f640e5e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:58.081531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:58.117671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22745c72-beb4-420b-bc54-dc734f640e5e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:58.122383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cef74-4fa5-46f7-bcf5-ed5b74ae2a7a None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.123060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cef74-4fa5-46f7-bcf5-ed5b74ae2a7a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.123060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cef74-4fa5-46f7-bcf5-ed5b74ae2a7a None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.124584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 458/915] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:43:58.129667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcb01-8dea-4545-9a38-1b72eed9a0f6 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.129955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcb01-8dea-4545-9a38-1b72eed9a0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.130233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcb01-8dea-4545-9a38-1b72eed9a0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.149560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2f2bcb01-8dea-4545-9a38-1b72eed9a0f6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:58.150768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2f2bcb01-8dea-4545-9a38-1b72eed9a0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:58.162394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b672379-be44-451f-b396-f055fab6eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:58.163430 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b672379-be44-451f-b396-f055fab6eb63 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.163908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b672379-be44-451f-b396-f055fab6eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.164425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b672379-be44-451f-b396-f055fab6eb63 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.165516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b672379-be44-451f-b396-f055fab6eb63 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:58.167207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b672379-be44-451f-b396-f055fab6eb63 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0b672379-be44-451f-b396-f055fab6eb63 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:58.177133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:58.178141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:43:58.179004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:58.186702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 458/916] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:43:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:58.188337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b672379-be44-451f-b396-f055fab6eb63 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:58.228901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 459/917] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:43:58.297183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b672379-be44-451f-b396-f055fab6eb63 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:58.334008 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25f4d1-c0c3-4b78-88a8-2d2e573b36a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:58.334995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25f4d1-c0c3-4b78-88a8-2d2e573b36a7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.335471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25f4d1-c0c3-4b78-88a8-2d2e573b36a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.335935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25f4d1-c0c3-4b78-88a8-2d2e573b36a7 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.336748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25f4d1-c0c3-4b78-88a8-2d2e573b36a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:58.337295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25f4d1-c0c3-4b78-88a8-2d2e573b36a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:58.338933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25f4d1-c0c3-4b78-88a8-2d2e573b36a7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:58.340115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:58.346080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 459/918] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:43:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:58.352440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bcb01-8dea-4545-9a38-1b72eed9a0f6 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:58.407919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:58.468367 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bcb01-8dea-4545-9a38-1b72eed9a0f6 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:58.500855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25f4d1-c0c3-4b78-88a8-2d2e573b36a7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:58.504186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c1196-b265-4f29-9dad-e79f47c9e636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:58.504826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c1196-b265-4f29-9dad-e79f47c9e636 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.505106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c1196-b265-4f29-9dad-e79f47c9e636 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.505380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c1196-b265-4f29-9dad-e79f47c9e636 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.506014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c1196-b265-4f29-9dad-e79f47c9e636 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:58.506315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c1196-b265-4f29-9dad-e79f47c9e636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:58.507686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c1196-b265-4f29-9dad-e79f47c9e636 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:58.514264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 460/919] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:43:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:58.514837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35caa0-5e0a-48f9-bdf5-20f3142f269f None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.515127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35caa0-5e0a-48f9-bdf5-20f3142f269f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.515562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35caa0-5e0a-48f9-bdf5-20f3142f269f None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.516898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 461/920] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:58.520075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35caa0-5e0a-48f9-bdf5-20f3142f269f None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:58.616502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25f4d1-c0c3-4b78-88a8-2d2e573b36a7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:58.632035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35caa0-5e0a-48f9-bdf5-20f3142f269f None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:58.645362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:58.654033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c1348-0d07-417d-a4cf-e28f6a207b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:58.654671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1348-0d07-417d-a4cf-e28f6a207b91 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.654955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1348-0d07-417d-a4cf-e28f6a207b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.655231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1348-0d07-417d-a4cf-e28f6a207b91 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.655812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c1348-0d07-417d-a4cf-e28f6a207b91 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:58.657331 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c1348-0d07-417d-a4cf-e28f6a207b91 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-531c1348-0d07-417d-a4cf-e28f6a207b91 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:58.667030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:58.668037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:43:58.669734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 460/921] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:43:58.670552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984ec92-4154-4b89-bccc-6e71382bf715 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.670552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984ec92-4154-4b89-bccc-6e71382bf715 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.670552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984ec92-4154-4b89-bccc-6e71382bf715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.676698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dbc7b-0add-4300-bb89-409a2b112277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:58.677340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dbc7b-0add-4300-bb89-409a2b112277 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.677618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dbc7b-0add-4300-bb89-409a2b112277 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.677887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dbc7b-0add-4300-bb89-409a2b112277 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.678578 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005dbc7b-0add-4300-bb89-409a2b112277 None admin] RBAC: Adding query filter params (interface=public, service_id=8fd28b5918bc47729f79bc9c2de7aa7e, region_id=RegionOne) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:58.678874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005dbc7b-0add-4300-bb89-409a2b112277 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:58.680231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005dbc7b-0add-4300-bb89-409a2b112277 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:58.685658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 462/922] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Sep 15 10:43:58 2022] GET /identity/v3/endpoints?service_id=8fd28b5918bc47729f79bc9c2de7aa7e&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:58.686424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dbc7b-0add-4300-bb89-409a2b112277 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:58.694816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5984ec92-4154-4b89-bccc-6e71382bf715 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:58.696364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5984ec92-4154-4b89-bccc-6e71382bf715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:58.759977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:58.780867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 461/923] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:58.780867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984ec92-4154-4b89-bccc-6e71382bf715 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:58.797611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dbc7b-0add-4300-bb89-409a2b112277 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:58.834489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9ef3c-6d41-4fd2-ae3b-100d97eb3851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:58.835009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9ef3c-6d41-4fd2-ae3b-100d97eb3851 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.835387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9ef3c-6d41-4fd2-ae3b-100d97eb3851 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.835660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9ef3c-6d41-4fd2-ae3b-100d97eb3851 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.836283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9ef3c-6d41-4fd2-ae3b-100d97eb3851 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:58.836585 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9ef3c-6d41-4fd2-ae3b-100d97eb3851 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:58.837983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9ef3c-6d41-4fd2-ae3b-100d97eb3851 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:58.843733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 463/924] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:58.849328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9ef3c-6d41-4fd2-ae3b-100d97eb3851 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:58.895596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:58.919314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984ec92-4154-4b89-bccc-6e71382bf715 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:58.962162 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9ef3c-6d41-4fd2-ae3b-100d97eb3851 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:58.967373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e418f1a-aa35-44a1-bb35-87c1876e35c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:58.968221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e418f1a-aa35-44a1-bb35-87c1876e35c2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:58.968711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e418f1a-aa35-44a1-bb35-87c1876e35c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:58.969187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e418f1a-aa35-44a1-bb35-87c1876e35c2 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:58.969972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e418f1a-aa35-44a1-bb35-87c1876e35c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:58.972087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e418f1a-aa35-44a1-bb35-87c1876e35c2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:58.985171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 462/925] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Sep 15 10:43:58 2022] GET /identity/v3/services => generated 1497 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:43:59.010088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518285b-eb10-4e78-bed1-85132b5f4041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:59.010712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518285b-eb10-4e78-bed1-85132b5f4041 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:59.010988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518285b-eb10-4e78-bed1-85132b5f4041 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:59.011254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518285b-eb10-4e78-bed1-85132b5f4041 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:59.011901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518285b-eb10-4e78-bed1-85132b5f4041 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:59.012432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518285b-eb10-4e78-bed1-85132b5f4041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:59.013877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518285b-eb10-4e78-bed1-85132b5f4041 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:59.025100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 464/926] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Sep 15 10:43:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:59.026530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e418f1a-aa35-44a1-bb35-87c1876e35c2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:59.084353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518285b-eb10-4e78-bed1-85132b5f4041 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:59.097901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:59.161687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e418f1a-aa35-44a1-bb35-87c1876e35c2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:59.205746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518285b-eb10-4e78-bed1-85132b5f4041 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:59.206265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ba05-c0e0-4abe-8e57-59ff562ac14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:59.206754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ba05-c0e0-4abe-8e57-59ff562ac14a None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:59.207030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ba05-c0e0-4abe-8e57-59ff562ac14a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:59.207297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ba05-c0e0-4abe-8e57-59ff562ac14a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:59.207958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9ba05-c0e0-4abe-8e57-59ff562ac14a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:59.209362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9ba05-c0e0-4abe-8e57-59ff562ac14a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:59.216520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:59.236528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 463/927] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:43:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:59.255669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afb038-e10d-45bb-af60-fadb357d2773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:59.256435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb038-e10d-45bb-af60-fadb357d2773 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:59.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb038-e10d-45bb-af60-fadb357d2773 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:59.257164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb038-e10d-45bb-af60-fadb357d2773 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:59.257833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afb038-e10d-45bb-af60-fadb357d2773 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:59.259303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afb038-e10d-45bb-af60-fadb357d2773 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-40afb038-e10d-45bb-af60-fadb357d2773 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:43:59.273731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:43:59.274782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:43:59.275999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 465/928] 10.222.0.50 () {56 vars in 1132 bytes} [Thu Sep 15 10:43:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:43:59.284724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ba05-c0e0-4abe-8e57-59ff562ac14a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:59.414577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0d71c-e081-4be4-958e-26bb53b19f89 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:59.414863 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0d71c-e081-4be4-958e-26bb53b19f89 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:59.415146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0d71c-e081-4be4-958e-26bb53b19f89 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:59.416768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 466/929] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:43:59.417017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ba05-c0e0-4abe-8e57-59ff562ac14a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:59.422955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82662fe-0bcd-41a8-8cc3-24bf1cfe3d2a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:59.423366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82662fe-0bcd-41a8-8cc3-24bf1cfe3d2a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:59.423738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82662fe-0bcd-41a8-8cc3-24bf1cfe3d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:59.432021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:59.452358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d82662fe-0bcd-41a8-8cc3-24bf1cfe3d2a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:43:59.453721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d82662fe-0bcd-41a8-8cc3-24bf1cfe3d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:43:59.461279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23e5f0-281b-46d9-bdff-09e42b505a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:59.461899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23e5f0-281b-46d9-bdff-09e42b505a52 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:59.462193 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23e5f0-281b-46d9-bdff-09e42b505a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:59.462882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23e5f0-281b-46d9-bdff-09e42b505a52 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:59.463515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23e5f0-281b-46d9-bdff-09e42b505a52 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:59.463815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23e5f0-281b-46d9-bdff-09e42b505a52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:59.465897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23e5f0-281b-46d9-bdff-09e42b505a52 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:59.479060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 464/930] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:43:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:43:59.484656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23e5f0-281b-46d9-bdff-09e42b505a52 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:59.538972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 467/931] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:43:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:43:59.595861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23e5f0-281b-46d9-bdff-09e42b505a52 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:59.631915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ec4be-9d55-4b94-98d0-cbec9c3c210c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:59.632535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec4be-9d55-4b94-98d0-cbec9c3c210c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:59.632829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec4be-9d55-4b94-98d0-cbec9c3c210c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:59.633129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec4be-9d55-4b94-98d0-cbec9c3c210c None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:59.633758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ec4be-9d55-4b94-98d0-cbec9c3c210c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:43:59.634064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ec4be-9d55-4b94-98d0-cbec9c3c210c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:59.635459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ec4be-9d55-4b94-98d0-cbec9c3c210c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:59.641479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 465/932] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:43:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:43:59.646989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82662fe-0bcd-41a8-8cc3-24bf1cfe3d2a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:59.749250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:59.752013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82662fe-0bcd-41a8-8cc3-24bf1cfe3d2a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:43:59.789058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939524b-21c1-4162-b96f-829330bb1bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:43:59.789670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939524b-21c1-4162-b96f-829330bb1bbc None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:43:59.789957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939524b-21c1-4162-b96f-829330bb1bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:43:59.790226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939524b-21c1-4162-b96f-829330bb1bbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:43:59.790967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939524b-21c1-4162-b96f-829330bb1bbc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:43:59.792348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939524b-21c1-4162-b96f-829330bb1bbc None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:43:59.816846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 468/933] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:43:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:43:59.852411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ec4be-9d55-4b94-98d0-cbec9c3c210c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:43:59.908251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:43:59.977174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ec4be-9d55-4b94-98d0-cbec9c3c210c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:00.020501 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e530147-7545-46ad-b664-c92b287ba34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:00.021096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e530147-7545-46ad-b664-c92b287ba34e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:00.021373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e530147-7545-46ad-b664-c92b287ba34e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:00.021640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e530147-7545-46ad-b664-c92b287ba34e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:00.035129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e530147-7545-46ad-b664-c92b287ba34e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6e530147-7545-46ad-b664-c92b287ba34e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:00.040854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:00.041881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:00.042750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 466/934] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Sep 15 10:43:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:00.048865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939524b-21c1-4162-b96f-829330bb1bbc None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:00.165165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939524b-21c1-4162-b96f-829330bb1bbc None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:00.202673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9685ec9-c526-47a4-bfc7-6db45ca9f746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:00.203282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9685ec9-c526-47a4-bfc7-6db45ca9f746 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:00.203557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9685ec9-c526-47a4-bfc7-6db45ca9f746 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:00.203874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9685ec9-c526-47a4-bfc7-6db45ca9f746 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:00.204533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9685ec9-c526-47a4-bfc7-6db45ca9f746 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:00.204849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9685ec9-c526-47a4-bfc7-6db45ca9f746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:00.206330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9685ec9-c526-47a4-bfc7-6db45ca9f746 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:00.215806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 469/935] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Sep 15 10:44:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:00.221269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e530147-7545-46ad-b664-c92b287ba34e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:00.326329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:00.332323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e530147-7545-46ad-b664-c92b287ba34e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:00.368238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a8822-29c7-4ac6-b74c-7b98d1dc6b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:00.368897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a8822-29c7-4ac6-b74c-7b98d1dc6b54 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:00.369210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a8822-29c7-4ac6-b74c-7b98d1dc6b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:00.369481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a8822-29c7-4ac6-b74c-7b98d1dc6b54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:00.382216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a8822-29c7-4ac6-b74c-7b98d1dc6b54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:00.383709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a8822-29c7-4ac6-b74c-7b98d1dc6b54 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-7b1a8822-29c7-4ac6-b74c-7b98d1dc6b54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:00.398726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:00.399799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:00.400722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:00.400722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:00.400722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:00.400722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 467/936] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:00.405902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9685ec9-c526-47a4-bfc7-6db45ca9f746 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:00.419881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:00.514262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9685ec9-c526-47a4-bfc7-6db45ca9f746 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:00.553311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3402d9-ba04-46b7-848f-77f51ec14d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:00.553918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3402d9-ba04-46b7-848f-77f51ec14d68 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:00.554192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3402d9-ba04-46b7-848f-77f51ec14d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:00.554514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3402d9-ba04-46b7-848f-77f51ec14d68 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:00.555157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3402d9-ba04-46b7-848f-77f51ec14d68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:00.555455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3402d9-ba04-46b7-848f-77f51ec14d68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:00.556880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3402d9-ba04-46b7-848f-77f51ec14d68 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:00.570199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 470/937] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:00.576430 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a8822-29c7-4ac6-b74c-7b98d1dc6b54 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:00.688080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a8822-29c7-4ac6-b74c-7b98d1dc6b54 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:00.722700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0585525-ac34-4365-87c3-9a6f857ed119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:00.723336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0585525-ac34-4365-87c3-9a6f857ed119 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:00.723614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0585525-ac34-4365-87c3-9a6f857ed119 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:00.723885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0585525-ac34-4365-87c3-9a6f857ed119 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:00.732473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0585525-ac34-4365-87c3-9a6f857ed119 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:00.733942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0585525-ac34-4365-87c3-9a6f857ed119 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-f0585525-ac34-4365-87c3-9a6f857ed119 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:44:00.745398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:00.746377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:00.747831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 468/938] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:44:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:00.754561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3402d9-ba04-46b7-848f-77f51ec14d68 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:00.828878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:00.864523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3402d9-ba04-46b7-848f-77f51ec14d68 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:00.899378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adc0a6-3d47-4807-a7aa-f3fa8bc09a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:00.899978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adc0a6-3d47-4807-a7aa-f3fa8bc09a02 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:00.900253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adc0a6-3d47-4807-a7aa-f3fa8bc09a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:00.900526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adc0a6-3d47-4807-a7aa-f3fa8bc09a02 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:00.901276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adc0a6-3d47-4807-a7aa-f3fa8bc09a02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:00.901576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adc0a6-3d47-4807-a7aa-f3fa8bc09a02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:00.903979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adc0a6-3d47-4807-a7aa-f3fa8bc09a02 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:00.923854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 471/939] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Sep 15 10:44:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:00.928512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0585525-ac34-4365-87c3-9a6f857ed119 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:00.940915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:01.033689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0585525-ac34-4365-87c3-9a6f857ed119 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:01.069946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0d59-1bba-45d6-912e-bb4f4c59b44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:01.070577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0d59-1bba-45d6-912e-bb4f4c59b44a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:01.070910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0d59-1bba-45d6-912e-bb4f4c59b44a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:01.071181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0d59-1bba-45d6-912e-bb4f4c59b44a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:01.083234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d0d59-1bba-45d6-912e-bb4f4c59b44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:01.084629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d0d59-1bba-45d6-912e-bb4f4c59b44a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-200d0d59-1bba-45d6-912e-bb4f4c59b44a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:01.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:01.102093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:01.104275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:01.104275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:01.104275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:01.108824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 469/940] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:01.109028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adc0a6-3d47-4807-a7aa-f3fa8bc09a02 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:01.212725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adc0a6-3d47-4807-a7aa-f3fa8bc09a02 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:01.241674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:01.251843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72136fc2-eb97-4f1e-9148-4b4e7e654d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:01.252453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72136fc2-eb97-4f1e-9148-4b4e7e654d11 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:01.252731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72136fc2-eb97-4f1e-9148-4b4e7e654d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:01.253025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72136fc2-eb97-4f1e-9148-4b4e7e654d11 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:01.253664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72136fc2-eb97-4f1e-9148-4b4e7e654d11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:01.253964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72136fc2-eb97-4f1e-9148-4b4e7e654d11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:01.255337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72136fc2-eb97-4f1e-9148-4b4e7e654d11 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:01.272311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 472/941] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:01.273678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0d59-1bba-45d6-912e-bb4f4c59b44a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:01.346831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:01.385068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0d59-1bba-45d6-912e-bb4f4c59b44a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:01.420954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bef20-df67-4709-98d1-3311a8870867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:01.421594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bef20-df67-4709-98d1-3311a8870867 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:01.421870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bef20-df67-4709-98d1-3311a8870867 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:01.422140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bef20-df67-4709-98d1-3311a8870867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:01.429104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967bef20-df67-4709-98d1-3311a8870867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:01.430553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967bef20-df67-4709-98d1-3311a8870867 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-967bef20-df67-4709-98d1-3311a8870867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:44:01.442243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:01.444308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:01.446485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:44:01.446485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:44:01.446485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:44:01.446485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:44:01.446485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:44:01.446485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:44:01.446485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:01.446485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:01.452084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 470/942] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:44:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:01.452292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72136fc2-eb97-4f1e-9148-4b4e7e654d11 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:01.459780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2fb20-7be4-40e5-85e9-2467599bf6b6 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:01.459780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2fb20-7be4-40e5-85e9-2467599bf6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:01.459780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2fb20-7be4-40e5-85e9-2467599bf6b6 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:01.459780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 471/943] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:01.467606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a0e8f-c5f0-4216-b778-a02ffb481249 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:01.467912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a0e8f-c5f0-4216-b778-a02ffb481249 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:01.468173 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a0e8f-c5f0-4216-b778-a02ffb481249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:01.488163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f64a0e8f-c5f0-4216-b778-a02ffb481249 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:01.489779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f64a0e8f-c5f0-4216-b778-a02ffb481249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:01.580382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72136fc2-eb97-4f1e-9148-4b4e7e654d11 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:01.581789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 472/944] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:44:01.598557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5f9f-2b9f-44af-912f-2b67b26a3d38 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:01.598832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5f9f-2b9f-44af-912f-2b67b26a3d38 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:01.599029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5f9f-2b9f-44af-912f-2b67b26a3d38 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:01.600494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 473/945] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:44:01.605600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568c8f1-b90c-4c56-8909-d0f8026b7361 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:01.606697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568c8f1-b90c-4c56-8909-d0f8026b7361 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:01.607192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568c8f1-b90c-4c56-8909-d0f8026b7361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:01.627053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d568c8f1-b90c-4c56-8909-d0f8026b7361 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:01.628572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d568c8f1-b90c-4c56-8909-d0f8026b7361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:01.630691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd0892-f737-429b-89f7-6e23c9dbf8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:01.631600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd0892-f737-429b-89f7-6e23c9dbf8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:01.631885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd0892-f737-429b-89f7-6e23c9dbf8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:01.632156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd0892-f737-429b-89f7-6e23c9dbf8c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:01.633743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd0892-f737-429b-89f7-6e23c9dbf8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:01.634053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd0892-f737-429b-89f7-6e23c9dbf8c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:01.635478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd0892-f737-429b-89f7-6e23c9dbf8c8 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:01.656884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 473/946] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:44:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:01.659383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd0892-f737-429b-89f7-6e23c9dbf8c8 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:01.685894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:01.716944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 474/947] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:44:01.723963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:01.779744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd0892-f737-429b-89f7-6e23c9dbf8c8 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:01.814915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a144e5e-77c1-4d57-adbb-58b4214b8e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:01.815572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a144e5e-77c1-4d57-adbb-58b4214b8e04 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:01.815848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a144e5e-77c1-4d57-adbb-58b4214b8e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:01.816117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a144e5e-77c1-4d57-adbb-58b4214b8e04 None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/0991767dd90d4f62bfe5484aad435669/roles/b4c8a5ff4e0c44e1a0bd3beaff37e4b4` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:01.818269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a144e5e-77c1-4d57-adbb-58b4214b8e04 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=0991767dd90d4f62bfe5484aad435669, role_id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:01.819698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a144e5e-77c1-4d57-adbb-58b4214b8e04 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:01.841007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 474/948] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:44:01 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/0991767dd90d4f62bfe5484aad435669/roles/b4c8a5ff4e0c44e1a0bd3beaff37e4b4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:44:01.858792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568c8f1-b90c-4c56-8909-d0f8026b7361 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:01.985765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568c8f1-b90c-4c56-8909-d0f8026b7361 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:02.000058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a144e5e-77c1-4d57-adbb-58b4214b8e04 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:02.029703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97156f15-bb6f-4ba7-87bc-09e17637bd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:02.030347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97156f15-bb6f-4ba7-87bc-09e17637bd13 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.030621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97156f15-bb6f-4ba7-87bc-09e17637bd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.030891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97156f15-bb6f-4ba7-87bc-09e17637bd13 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.031476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97156f15-bb6f-4ba7-87bc-09e17637bd13 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:02.032892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97156f15-bb6f-4ba7-87bc-09e17637bd13 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-97156f15-bb6f-4ba7-87bc-09e17637bd13 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:02.041899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:02.042945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:02.044515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 475/949] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:44:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:02.049777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97156f15-bb6f-4ba7-87bc-09e17637bd13 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:02.106367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:02.146960 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a144e5e-77c1-4d57-adbb-58b4214b8e04 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:02.159241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:02.187370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97156f15-bb6f-4ba7-87bc-09e17637bd13 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:02.194190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0c891-b134-4c9f-845b-c59b6c74b288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:02.194845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0c891-b134-4c9f-845b-c59b6c74b288 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.195126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0c891-b134-4c9f-845b-c59b6c74b288 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.195395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0c891-b134-4c9f-845b-c59b6c74b288 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.196589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0c891-b134-4c9f-845b-c59b6c74b288 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:02.198260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0c891-b134-4c9f-845b-c59b6c74b288 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:02.203264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WARNING keystone.server.flask.common [None req-bcc0c891-b134-4c9f-845b-c59b6c74b288 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 10:44:02.228423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.sql.core [None req-bcc0c891-b134-4c9f-845b-c59b6c74b288 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 10:44:02.228423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 10:44:02.228423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [parameters: {'id': '740e2a40f1154d09aa2bcb52d79725af', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 10:44:02.228423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81388) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 10:44:02.229406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246640b-14f4-4505-bb7e-90aec0d808f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:02.230237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246640b-14f4-4505-bb7e-90aec0d808f4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.231401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246640b-14f4-4505-bb7e-90aec0d808f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.231928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246640b-14f4-4505-bb7e-90aec0d808f4 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.232789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246640b-14f4-4505-bb7e-90aec0d808f4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:02.233292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246640b-14f4-4505-bb7e-90aec0d808f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:02.234914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246640b-14f4-4505-bb7e-90aec0d808f4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:02.240232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 476/950] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:44:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:02.241696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d771512-6401-4088-9e85-8e80bf2f2e7a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.242184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d771512-6401-4088-9e85-8e80bf2f2e7a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.242645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d771512-6401-4088-9e85-8e80bf2f2e7a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.244275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 477/951] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:02.246205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d771512-6401-4088-9e85-8e80bf2f2e7a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-bcc0c891-b134-4c9f-845b-c59b6c74b288 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 740e2a40f1154d09aa2bcb52d79725af.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 740e2a40f1154d09aa2bcb52d79725af. Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application conn.query(q) Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 10:44:02.248437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application result.read() Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 15 10:44:02.249327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application next(self.gen) Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application yield resource Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application next(self.gen) Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise self.value Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application session.commit() Sep 15 10:44:02.250309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self.session.flush() Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application compat.raise_( Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 10:44:02.251242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 10:44:02.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application conn.query(q) Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application result.read() Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 10:44:02.254072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [parameters: {'id': '740e2a40f1154d09aa2bcb52d79725af', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:02.255758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:02.256789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 740e2a40f1154d09aa2bcb52d79725af. Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 475/952] 10.222.0.50 () {60 vars in 1140 bytes} [Thu Sep 15 10:44:01 2022] POST /identity/v3/projects => generated 262 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640fab3-4cb0-4b03-bace-8cba223160a7 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.259048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640fab3-4cb0-4b03-bace-8cba223160a7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.260706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640fab3-4cb0-4b03-bace-8cba223160a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.274145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-f640fab3-4cb0-4b03-bace-8cba223160a7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:02.275687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f640fab3-4cb0-4b03-bace-8cba223160a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:02.348848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 476/953] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:02.349058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640fab3-4cb0-4b03-bace-8cba223160a7 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:02.375389 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d771512-6401-4088-9e85-8e80bf2f2e7a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:02.391821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:02.427785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c955e7-dbd5-449a-8f06-2a7258b69dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:02.428758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c955e7-dbd5-449a-8f06-2a7258b69dcd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.429251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c955e7-dbd5-449a-8f06-2a7258b69dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.429706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c955e7-dbd5-449a-8f06-2a7258b69dcd None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.430905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c955e7-dbd5-449a-8f06-2a7258b69dcd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:02.431424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c955e7-dbd5-449a-8f06-2a7258b69dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:02.434575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c955e7-dbd5-449a-8f06-2a7258b69dcd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:02.443623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 478/954] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:44:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:02.450366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c955e7-dbd5-449a-8f06-2a7258b69dcd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:02.465277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:02.476608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640fab3-4cb0-4b03-bace-8cba223160a7 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:02.525101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2852d93-dbf1-4fe6-84d6-aa896669cd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:02.525745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2852d93-dbf1-4fe6-84d6-aa896669cd84 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.526093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2852d93-dbf1-4fe6-84d6-aa896669cd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.526376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2852d93-dbf1-4fe6-84d6-aa896669cd84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.536888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2852d93-dbf1-4fe6-84d6-aa896669cd84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:02.543906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2852d93-dbf1-4fe6-84d6-aa896669cd84 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-c2852d93-dbf1-4fe6-84d6-aa896669cd84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:44:02.560501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:02.561590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:02.562603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:44:02.562603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:44:02.562603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:44:02.562603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:44:02.562603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:44:02.562603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:44:02.562603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:02.562603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:02.564951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 477/955] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:02.567203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2852d93-dbf1-4fe6-84d6-aa896669cd84 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:02.578283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c955e7-dbd5-449a-8f06-2a7258b69dcd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:02.619170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee88ed0-9b79-423a-9ba2-3f0f792db460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:02.619799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88ed0-9b79-423a-9ba2-3f0f792db460 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.620096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88ed0-9b79-423a-9ba2-3f0f792db460 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.620359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88ed0-9b79-423a-9ba2-3f0f792db460 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.621150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee88ed0-9b79-423a-9ba2-3f0f792db460 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:02.623239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee88ed0-9b79-423a-9ba2-3f0f792db460 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:02.644889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 479/956] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:44:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:02.650594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee88ed0-9b79-423a-9ba2-3f0f792db460 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:02.685067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2852d93-dbf1-4fe6-84d6-aa896669cd84 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:02.694938 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:02.748564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcf9eb-f8aa-40ea-8494-e3f6926db8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:02.749661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcf9eb-f8aa-40ea-8494-e3f6926db8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.750442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcf9eb-f8aa-40ea-8494-e3f6926db8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.750929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcf9eb-f8aa-40ea-8494-e3f6926db8f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.752599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcf9eb-f8aa-40ea-8494-e3f6926db8f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:02.753448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcf9eb-f8aa-40ea-8494-e3f6926db8f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:02.756466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcf9eb-f8aa-40ea-8494-e3f6926db8f7 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:02.783988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 478/957] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:44:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:02.802704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee88ed0-9b79-423a-9ba2-3f0f792db460 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:02.850689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc11136-523e-40d6-95c8-a21190e822ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:02.851361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc11136-523e-40d6-95c8-a21190e822ed None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:02.851645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc11136-523e-40d6-95c8-a21190e822ed None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:02.851920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc11136-523e-40d6-95c8-a21190e822ed None admin] PATH_INFO: `/v3/services/compute` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:02.852556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc11136-523e-40d6-95c8-a21190e822ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:02.854030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc11136-523e-40d6-95c8-a21190e822ed None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2fc11136-523e-40d6-95c8-a21190e822ed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:02.864591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:02.865833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:02.866935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 480/958] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:44:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:02.875248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcf9eb-f8aa-40ea-8494-e3f6926db8f7 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:02.892020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:03.001477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcf9eb-f8aa-40ea-8494-e3f6926db8f7 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:03.037894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585617f-eaa8-4179-8625-eb70208495b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:03.038509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585617f-eaa8-4179-8625-eb70208495b0 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:03.038792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585617f-eaa8-4179-8625-eb70208495b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:03.039063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585617f-eaa8-4179-8625-eb70208495b0 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:03.039699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585617f-eaa8-4179-8625-eb70208495b0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:03.040002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585617f-eaa8-4179-8625-eb70208495b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:03.041413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585617f-eaa8-4179-8625-eb70208495b0 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:03.047703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 479/959] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:44:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:03.053723 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc11136-523e-40d6-95c8-a21190e822ed None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:03.158948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:03.169290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc11136-523e-40d6-95c8-a21190e822ed None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:03.207375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df857a-e7f1-4f0a-bf69-2ba1fe96fb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:03.208611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df857a-e7f1-4f0a-bf69-2ba1fe96fb46 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:03.209148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df857a-e7f1-4f0a-bf69-2ba1fe96fb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:03.209666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df857a-e7f1-4f0a-bf69-2ba1fe96fb46 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:03.210490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df857a-e7f1-4f0a-bf69-2ba1fe96fb46 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:03.211344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df857a-e7f1-4f0a-bf69-2ba1fe96fb46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:03.213015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df857a-e7f1-4f0a-bf69-2ba1fe96fb46 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:03.219359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 481/960] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:44:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:04.155292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb91c94-c753-4cfe-826b-2a65f6607bf2 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:04.155810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb91c94-c753-4cfe-826b-2a65f6607bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:04.155810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb91c94-c753-4cfe-826b-2a65f6607bf2 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:04.157432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 480/961] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:04.162718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8784f19-e14b-4559-b279-49459cfa0c94 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:04.163109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8784f19-e14b-4559-b279-49459cfa0c94 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:04.163276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8784f19-e14b-4559-b279-49459cfa0c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:04.181661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d8784f19-e14b-4559-b279-49459cfa0c94 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:04.182845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d8784f19-e14b-4559-b279-49459cfa0c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:04.254177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 482/962] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:04.526216 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb91c94-c753-4cfe-826b-2a65f6607bf2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:04.623092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:04.654576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb91c94-c753-4cfe-826b-2a65f6607bf2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:04.690606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36719a3e-2c8d-4055-8971-c5e017279d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:04.691482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36719a3e-2c8d-4055-8971-c5e017279d75 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:04.691918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36719a3e-2c8d-4055-8971-c5e017279d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:04.692370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36719a3e-2c8d-4055-8971-c5e017279d75 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:04.704459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36719a3e-2c8d-4055-8971-c5e017279d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-36719a3e-2c8d-4055-8971-c5e017279d75 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:04.709567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:04.710712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:04.711792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:04.714100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 481/963] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Sep 15 10:44:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:04.720263 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8784f19-e14b-4559-b279-49459cfa0c94 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:04.813937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:04.830892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8784f19-e14b-4559-b279-49459cfa0c94 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:04.874852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8aa9a-12d0-42ba-873b-6e070659c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:04.875788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8aa9a-12d0-42ba-873b-6e070659c3af None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:04.876248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8aa9a-12d0-42ba-873b-6e070659c3af None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:04.876723 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8aa9a-12d0-42ba-873b-6e070659c3af None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:04.877636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8aa9a-12d0-42ba-873b-6e070659c3af None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:04.878115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8aa9a-12d0-42ba-873b-6e070659c3af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:04.879663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8aa9a-12d0-42ba-873b-6e070659c3af None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:04.889222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 483/964] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Sep 15 10:44:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:04.894702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36719a3e-2c8d-4055-8971-c5e017279d75 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:04.962417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfb329-5c0d-4313-8c9f-9770fbf4d58b None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:04.962743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfb329-5c0d-4313-8c9f-9770fbf4d58b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:04.962984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfb329-5c0d-4313-8c9f-9770fbf4d58b None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:04.964582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 484/965] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:44:04.970375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f262f-53c0-485c-9c23-b735b1086969 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:04.970664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f262f-53c0-485c-9c23-b735b1086969 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:04.970946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f262f-53c0-485c-9c23-b735b1086969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:04.990313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c24f262f-53c0-485c-9c23-b735b1086969 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:04.991646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c24f262f-53c0-485c-9c23-b735b1086969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:05.003839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36719a3e-2c8d-4055-8971-c5e017279d75 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:05.039646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f8bf9-474e-4d46-8f65-bdb681f81281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:05.040277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f8bf9-474e-4d46-8f65-bdb681f81281 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.040651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f8bf9-474e-4d46-8f65-bdb681f81281 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.040952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f8bf9-474e-4d46-8f65-bdb681f81281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.042853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:05.053579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f8bf9-474e-4d46-8f65-bdb681f81281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:05.055009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f8bf9-474e-4d46-8f65-bdb681f81281 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-481f8bf9-474e-4d46-8f65-bdb681f81281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:05.077053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:05.078066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:05.079056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:05.079056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:05.079056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:05.079056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 482/967] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:05.079300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 485/967] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:44:05.082973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f8bf9-474e-4d46-8f65-bdb681f81281 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:05.187138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f8bf9-474e-4d46-8f65-bdb681f81281 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:05.196323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5bfac-06a0-4c3a-890b-33484fd3068e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.196774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5bfac-06a0-4c3a-890b-33484fd3068e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.196917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5bfac-06a0-4c3a-890b-33484fd3068e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.198409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 486/968] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:05.205130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6763e-cf1d-45b1-beef-5336e8a4f385 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.205130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6763e-cf1d-45b1-beef-5336e8a4f385 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.205130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6763e-cf1d-45b1-beef-5336e8a4f385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.225173 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f3d6763e-cf1d-45b1-beef-5336e8a4f385 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:05.226353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f3d6763e-cf1d-45b1-beef-5336e8a4f385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:05.229100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c66ab-8d6c-4faa-9143-bd8cb788fdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:05.229949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c66ab-8d6c-4faa-9143-bd8cb788fdf3 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.231131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c66ab-8d6c-4faa-9143-bd8cb788fdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.231669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c66ab-8d6c-4faa-9143-bd8cb788fdf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.232540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c66ab-8d6c-4faa-9143-bd8cb788fdf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:05.233146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c66ab-8d6c-4faa-9143-bd8cb788fdf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:05.234831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c66ab-8d6c-4faa-9143-bd8cb788fdf3 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:05.247678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 483/969] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:05.252770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c66ab-8d6c-4faa-9143-bd8cb788fdf3 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:05.308661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 487/970] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:05.309693 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:05.359742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c66ab-8d6c-4faa-9143-bd8cb788fdf3 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:05.374777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6763e-cf1d-45b1-beef-5336e8a4f385 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:05.386376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:05.400436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b322722-d33d-4cbe-8588-ff379448796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:05.401146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b322722-d33d-4cbe-8588-ff379448796a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.401412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b322722-d33d-4cbe-8588-ff379448796a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.401689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b322722-d33d-4cbe-8588-ff379448796a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.411354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b322722-d33d-4cbe-8588-ff379448796a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:05.412920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b322722-d33d-4cbe-8588-ff379448796a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-4b322722-d33d-4cbe-8588-ff379448796a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:44:05.426566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:05.427561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:05.428619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 484/971] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:44:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:05.435075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b322722-d33d-4cbe-8588-ff379448796a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:05.508550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6763e-cf1d-45b1-beef-5336e8a4f385 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:05.543760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393d038-2864-40dc-aff4-9e8718afacd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:05.544405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393d038-2864-40dc-aff4-9e8718afacd7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.544701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393d038-2864-40dc-aff4-9e8718afacd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.544993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393d038-2864-40dc-aff4-9e8718afacd7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.545581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393d038-2864-40dc-aff4-9e8718afacd7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:05.546969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393d038-2864-40dc-aff4-9e8718afacd7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:05.553795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b322722-d33d-4cbe-8588-ff379448796a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-8393d038-2864-40dc-aff4-9e8718afacd7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:05.555922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:05.556988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:05.565286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 488/972] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:44:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:05.566355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3b15e-60b4-4d4e-8570-01981b2d4eb7 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.566355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3b15e-60b4-4d4e-8570-01981b2d4eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.566355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3b15e-60b4-4d4e-8570-01981b2d4eb7 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.566355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 489/973] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:05.566355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3b15e-60b4-4d4e-8570-01981b2d4eb7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:05.574667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:05.606719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf48409-9b82-4210-9c56-1efe06f7fe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:05.607626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf48409-9b82-4210-9c56-1efe06f7fe02 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.608050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf48409-9b82-4210-9c56-1efe06f7fe02 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.608465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf48409-9b82-4210-9c56-1efe06f7fe02 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.609641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf48409-9b82-4210-9c56-1efe06f7fe02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:05.609998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf48409-9b82-4210-9c56-1efe06f7fe02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:05.611363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf48409-9b82-4210-9c56-1efe06f7fe02 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:05.630818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 485/974] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Sep 15 10:44:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:05.632081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0189-371f-4a8f-91d6-1a34fd8b83d3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.632367 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0189-371f-4a8f-91d6-1a34fd8b83d3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.632775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0189-371f-4a8f-91d6-1a34fd8b83d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.640622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:05.651869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2f1c0189-371f-4a8f-91d6-1a34fd8b83d3 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:05.653452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2f1c0189-371f-4a8f-91d6-1a34fd8b83d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:05.671632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3b15e-60b4-4d4e-8570-01981b2d4eb7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:05.706460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739c44b-a1ee-4dbb-8965-1e6e96bff626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:05.707068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739c44b-a1ee-4dbb-8965-1e6e96bff626 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.707350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739c44b-a1ee-4dbb-8965-1e6e96bff626 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.707630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739c44b-a1ee-4dbb-8965-1e6e96bff626 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.708252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739c44b-a1ee-4dbb-8965-1e6e96bff626 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:05.708552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739c44b-a1ee-4dbb-8965-1e6e96bff626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:05.709957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739c44b-a1ee-4dbb-8965-1e6e96bff626 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:05.715043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 490/975] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:44:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:05.715644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739c44b-a1ee-4dbb-8965-1e6e96bff626 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:05.722179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 486/976] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:05.723005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0189-371f-4a8f-91d6-1a34fd8b83d3 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:05.823611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:05.825077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739c44b-a1ee-4dbb-8965-1e6e96bff626 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:05.832053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0189-371f-4a8f-91d6-1a34fd8b83d3 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:05.861697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:05.862810 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.863094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.863368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.864001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:05.865481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:05.868016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING keystone.server.flask.common [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 10:44:05.870433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181964a2-13a9-4cac-bc37-3f59f8464944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:05.872002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181964a2-13a9-4cac-bc37-3f59f8464944 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:05.872634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181964a2-13a9-4cac-bc37-3f59f8464944 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:05.873129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181964a2-13a9-4cac-bc37-3f59f8464944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:05.877537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:05.881465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.sql.core [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 10:44:05.881465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 10:44:05.881465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [parameters: {'user_id': '837188c5728448d981928f8897219db6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 10:44:05.881465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81387) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application conn.query(q) Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result.read() Sep 15 10:44:05.892691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 10:44:05.893915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application next(self.gen) Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application yield resource Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application next(self.gen) Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise self.value Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application session.commit() Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 10:44:05.895115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.session.flush() Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application compat.raise_( Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 10:44:05.896289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 10:44:05.897429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application conn.query(q) Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result.read() Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 10:44:05.898400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [parameters: {'user_id': '837188c5728448d981928f8897219db6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:05.899265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 15 10:44:05.900081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 15 10:44:05.901550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 15 10:44:05.901550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 15 10:44:05.901550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 10:44:05.901550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 10:44:05.901550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 10:44:05.901550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:05.901550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 491/977] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Sep 15 10:44:05 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:05.901550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:05.959341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181964a2-13a9-4cac-bc37-3f59f8464944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:05.960928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181964a2-13a9-4cac-bc37-3f59f8464944 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.000570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5b52f-1256-4276-b13d-d2da0677fda3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:06.033981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3470b-2403-4381-a3d5-fe889f657430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:06.034611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3470b-2403-4381-a3d5-fe889f657430 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:06.034893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3470b-2403-4381-a3d5-fe889f657430 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:06.035164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3470b-2403-4381-a3d5-fe889f657430 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:06.035783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3470b-2403-4381-a3d5-fe889f657430 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:06.036083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3470b-2403-4381-a3d5-fe889f657430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:06.037681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3470b-2403-4381-a3d5-fe889f657430 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.045971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 492/978] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:44:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:06.046649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3470b-2403-4381-a3d5-fe889f657430 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:06.056719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:06.065378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 487/979] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:05 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:44:06.066293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181964a2-13a9-4cac-bc37-3f59f8464944 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:06.110600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:06.148163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3470b-2403-4381-a3d5-fe889f657430 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:06.166836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181964a2-13a9-4cac-bc37-3f59f8464944 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:06.182674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216c771-63c7-40ea-b7c2-066b4ef29a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:06.183306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216c771-63c7-40ea-b7c2-066b4ef29a72 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:06.183588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216c771-63c7-40ea-b7c2-066b4ef29a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:06.183857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216c771-63c7-40ea-b7c2-066b4ef29a72 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:06.191225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216c771-63c7-40ea-b7c2-066b4ef29a72 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:06.192665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216c771-63c7-40ea-b7c2-066b4ef29a72 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.201418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6448e-d8b0-414a-baf5-4190f7aaf018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:06.202023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6448e-d8b0-414a-baf5-4190f7aaf018 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:06.202301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6448e-d8b0-414a-baf5-4190f7aaf018 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:06.202571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6448e-d8b0-414a-baf5-4190f7aaf018 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:06.203453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6448e-d8b0-414a-baf5-4190f7aaf018 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3216c771-63c7-40ea-b7c2-066b4ef29a72 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:44:06.204615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:06.205952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 10:44:06.206918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:06.207479 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6448e-d8b0-414a-baf5-4190f7aaf018 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.207560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 493/980] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:44:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:06.207560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216c771-63c7-40ea-b7c2-066b4ef29a72 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:06.221286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 488/981] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:44:06 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:06.221286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6448e-d8b0-414a-baf5-4190f7aaf018 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:06.295918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:06.329067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216c771-63c7-40ea-b7c2-066b4ef29a72 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:06.354883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:06.358822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6448e-d8b0-414a-baf5-4190f7aaf018 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:06.368480 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75829180-0d67-490b-9dc0-aea060d7a2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:06.369133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75829180-0d67-490b-9dc0-aea060d7a2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:06.369458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75829180-0d67-490b-9dc0-aea060d7a2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:06.369973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75829180-0d67-490b-9dc0-aea060d7a2b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:06.370738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75829180-0d67-490b-9dc0-aea060d7a2b9 None admin] RBAC: Adding query filter params (interface=admin, service_id=8fd28b5918bc47729f79bc9c2de7aa7e, region_id=RegionOne) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:06.371112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75829180-0d67-490b-9dc0-aea060d7a2b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:06.372960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75829180-0d67-490b-9dc0-aea060d7a2b9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.383202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 494/982] 10.222.0.50 () {56 vars in 1249 bytes} [Thu Sep 15 10:44:06 2022] GET /identity/v3/endpoints?service_id=8fd28b5918bc47729f79bc9c2de7aa7e&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:06.383202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75829180-0d67-490b-9dc0-aea060d7a2b9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:06.404342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bde812-36e6-40fb-bbe6-b23bdec318e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:06.404946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bde812-36e6-40fb-bbe6-b23bdec318e3 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:06.405229 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bde812-36e6-40fb-bbe6-b23bdec318e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:06.405497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bde812-36e6-40fb-bbe6-b23bdec318e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:06.419104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bde812-36e6-40fb-bbe6-b23bdec318e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:06.421270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bde812-36e6-40fb-bbe6-b23bdec318e3 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-d4bde812-36e6-40fb-bbe6-b23bdec318e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:06.437606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:06.438611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:06.439518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:06.439518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:06.439518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:06.439518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 489/983] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:06.439518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bde812-36e6-40fb-bbe6-b23bdec318e3 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:06.495072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75829180-0d67-490b-9dc0-aea060d7a2b9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:06.529396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac7fce-acb3-428d-9b2a-15e216ee0c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:06.530011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac7fce-acb3-428d-9b2a-15e216ee0c0c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:06.530295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac7fce-acb3-428d-9b2a-15e216ee0c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:06.530570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac7fce-acb3-428d-9b2a-15e216ee0c0c None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:06.531259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac7fce-acb3-428d-9b2a-15e216ee0c0c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:06.531563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac7fce-acb3-428d-9b2a-15e216ee0c0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:06.533096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac7fce-acb3-428d-9b2a-15e216ee0c0c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.547014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bde812-36e6-40fb-bbe6-b23bdec318e3 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:06.548719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:06.571495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 495/984] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Sep 15 10:44:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:06.574043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac7fce-acb3-428d-9b2a-15e216ee0c0c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:06.596245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d277c-65ad-4035-9bf5-a9a9e1695b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:06.597111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d277c-65ad-4035-9bf5-a9a9e1695b70 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:06.597437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d277c-65ad-4035-9bf5-a9a9e1695b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:06.597813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d277c-65ad-4035-9bf5-a9a9e1695b70 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:06.598539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d277c-65ad-4035-9bf5-a9a9e1695b70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:06.600101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d277c-65ad-4035-9bf5-a9a9e1695b70 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.609318 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 490/985] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Sep 15 10:44:06 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:44:06.640123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:06.708794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac7fce-acb3-428d-9b2a-15e216ee0c0c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:06.757844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfc585-0c96-40e8-90fb-6d46d51727a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:06.758515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfc585-0c96-40e8-90fb-6d46d51727a3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:06.758805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfc585-0c96-40e8-90fb-6d46d51727a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:06.759079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfc585-0c96-40e8-90fb-6d46d51727a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:06.759770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfc585-0c96-40e8-90fb-6d46d51727a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:06.760072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfc585-0c96-40e8-90fb-6d46d51727a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:06.761638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfc585-0c96-40e8-90fb-6d46d51727a3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.776891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 496/986] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:06.782446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d277c-65ad-4035-9bf5-a9a9e1695b70 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:06.897284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d277c-65ad-4035-9bf5-a9a9e1695b70 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:06.931544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bceea-a0b6-49fa-9cc4-f02c4613e9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:06.932181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bceea-a0b6-49fa-9cc4-f02c4613e9fd None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:06.932460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bceea-a0b6-49fa-9cc4-f02c4613e9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:06.932734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bceea-a0b6-49fa-9cc4-f02c4613e9fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:06.940781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5bceea-a0b6-49fa-9cc4-f02c4613e9fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:06.942281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5bceea-a0b6-49fa-9cc4-f02c4613e9fd None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-ea5bceea-a0b6-49fa-9cc4-f02c4613e9fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:44:06.955244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:06.956218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:06.957272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:44:06.957272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:44:06.957272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:44:06.957272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:44:06.957272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:44:06.957272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:44:06.957272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:06.957272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:06.957272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 491/987] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:44:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:06.962262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfc585-0c96-40e8-90fb-6d46d51727a3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:07.065656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfc585-0c96-40e8-90fb-6d46d51727a3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:07.068006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:07.103048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c45ad6-849f-467a-962c-c8c189501338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:07.103658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c45ad6-849f-467a-962c-c8c189501338 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:07.103939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c45ad6-849f-467a-962c-c8c189501338 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:07.104211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c45ad6-849f-467a-962c-c8c189501338 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:07.104924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c45ad6-849f-467a-962c-c8c189501338 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:07.105233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c45ad6-849f-467a-962c-c8c189501338 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:07.106861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c45ad6-849f-467a-962c-c8c189501338 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:07.123243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 497/988] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:44:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:07.124824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bceea-a0b6-49fa-9cc4-f02c4613e9fd None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:07.157814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:07.235759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bceea-a0b6-49fa-9cc4-f02c4613e9fd None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:07.269704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c8d81-a51f-4adf-8c71-f66dba0484a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:07.270307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c8d81-a51f-4adf-8c71-f66dba0484a5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:07.270591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c8d81-a51f-4adf-8c71-f66dba0484a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:07.270866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c8d81-a51f-4adf-8c71-f66dba0484a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:07.271586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c8d81-a51f-4adf-8c71-f66dba0484a5 None admin] RBAC: Adding query filter params (role.id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=0991767dd90d4f62bfe5484aad435669) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:07.271900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c8d81-a51f-4adf-8c71-f66dba0484a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:07.273429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c8d81-a51f-4adf-8c71-f66dba0484a5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:07.288890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 492/989] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:44:07 2022] GET /identity/v3/role_assignments?user.id=0991767dd90d4f62bfe5484aad435669&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:08.439878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c684-ed66-40da-88e2-5b5d6200489f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:08.440495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c684-ed66-40da-88e2-5b5d6200489f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:08.440495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c684-ed66-40da-88e2-5b5d6200489f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:08.441978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 498/990] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:08.446932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311c3dd-da80-4244-814a-7c9212353f61 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:08.447321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311c3dd-da80-4244-814a-7c9212353f61 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:08.447499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311c3dd-da80-4244-814a-7c9212353f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:08.466450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-f311c3dd-da80-4244-814a-7c9212353f61 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:08.467638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f311c3dd-da80-4244-814a-7c9212353f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:08.537744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 493/991] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:08.810410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662c684-ed66-40da-88e2-5b5d6200489f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:08.843921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36976ac-3c81-43eb-8234-148fdc04e677 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:08.844214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36976ac-3c81-43eb-8234-148fdc04e677 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:08.844499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36976ac-3c81-43eb-8234-148fdc04e677 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:08.846014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 494/992] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:44:08.850787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e24081-8b71-4112-ae29-54cd32556a39 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:08.851072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e24081-8b71-4112-ae29-54cd32556a39 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:08.851352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e24081-8b71-4112-ae29-54cd32556a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:08.870200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-14e24081-8b71-4112-ae29-54cd32556a39 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:08.871944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-14e24081-8b71-4112-ae29-54cd32556a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:08.891105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:08.921464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662c684-ed66-40da-88e2-5b5d6200489f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:08.952852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 495/993] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:44:08.956877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7dde-ddeb-42a7-98f8-785d6c6daef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:08.957489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7dde-ddeb-42a7-98f8-785d6c6daef1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:08.957766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7dde-ddeb-42a7-98f8-785d6c6daef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:08.958040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7dde-ddeb-42a7-98f8-785d6c6daef1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:08.958609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee7dde-ddeb-42a7-98f8-785d6c6daef1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:08.959976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee7dde-ddeb-42a7-98f8-785d6c6daef1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-13ee7dde-ddeb-42a7-98f8-785d6c6daef1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:08.969402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:08.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:08.973840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 499/994] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:44:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:08.976357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e24081-8b71-4112-ae29-54cd32556a39 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:09.079741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e24081-8b71-4112-ae29-54cd32556a39 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:09.114530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7d0f-5ca8-4440-9411-cab9fbfda9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:09.115137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7d0f-5ca8-4440-9411-cab9fbfda9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.115413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7d0f-5ca8-4440-9411-cab9fbfda9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.115682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7d0f-5ca8-4440-9411-cab9fbfda9c7 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.116303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c7d0f-5ca8-4440-9411-cab9fbfda9c7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:09.116605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c7d0f-5ca8-4440-9411-cab9fbfda9c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:09.118005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c7d0f-5ca8-4440-9411-cab9fbfda9c7 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:09.127492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 496/995] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:44:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:09.128495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7dde-ddeb-42a7-98f8-785d6c6daef1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:09.158802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:09.222097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7d0f-5ca8-4440-9411-cab9fbfda9c7 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:09.235534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7dde-ddeb-42a7-98f8-785d6c6daef1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:09.238369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:09.275659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ec791-e681-48a8-823d-bae776403750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:09.276265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ec791-e681-48a8-823d-bae776403750 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.276544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ec791-e681-48a8-823d-bae776403750 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.276829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ec791-e681-48a8-823d-bae776403750 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.277795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ec791-e681-48a8-823d-bae776403750 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:09.278181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ec791-e681-48a8-823d-bae776403750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:09.279567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ec791-e681-48a8-823d-bae776403750 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:09.284951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 500/996] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:44:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:09.290410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ec791-e681-48a8-823d-bae776403750 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:09.341168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7d0f-5ca8-4440-9411-cab9fbfda9c7 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:09.384945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3f196-7856-4717-b944-602b4d910295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:09.385599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3f196-7856-4717-b944-602b4d910295 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.385875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3f196-7856-4717-b944-602b4d910295 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.386142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3f196-7856-4717-b944-602b4d910295 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.388127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3f196-7856-4717-b944-602b4d910295 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:09.389446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3f196-7856-4717-b944-602b4d910295 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:09.393155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-46f3f196-7856-4717-b944-602b4d910295 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:09.399507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:09.400565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:09.401487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 497/997] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Sep 15 10:44:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:09.407242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3f196-7856-4717-b944-602b4d910295 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:09.409273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ec791-e681-48a8-823d-bae776403750 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:09.447168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a317f-7993-4c13-af94-3ed09aea558d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:09.447839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a317f-7993-4c13-af94-3ed09aea558d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.448092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a317f-7993-4c13-af94-3ed09aea558d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.448655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a317f-7993-4c13-af94-3ed09aea558d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.449062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a317f-7993-4c13-af94-3ed09aea558d None admin] RBAC: Adding query filter params (interface=public, service_id=6bfd368615524552bead1e590f55ddbf, region_id=RegionOne) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:09.449372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a317f-7993-4c13-af94-3ed09aea558d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:09.451019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a317f-7993-4c13-af94-3ed09aea558d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:09.456669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 501/998] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Sep 15 10:44:09 2022] GET /identity/v3/endpoints?service_id=6bfd368615524552bead1e590f55ddbf&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:09.462024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a317f-7993-4c13-af94-3ed09aea558d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:09.491553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:09.514291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3f196-7856-4717-b944-602b4d910295 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:09.549486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44662085-4859-496c-ba0c-4b5de05fe070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:09.550091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44662085-4859-496c-ba0c-4b5de05fe070 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.550368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44662085-4859-496c-ba0c-4b5de05fe070 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.550636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44662085-4859-496c-ba0c-4b5de05fe070 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.551269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44662085-4859-496c-ba0c-4b5de05fe070 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:09.551567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44662085-4859-496c-ba0c-4b5de05fe070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:09.555094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44662085-4859-496c-ba0c-4b5de05fe070 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:09.558011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 498/999] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:44:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:09.559173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d1b4c-6fb2-4ff8-80d7-d632512bf8b2 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.559460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d1b4c-6fb2-4ff8-80d7-d632512bf8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.559730 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d1b4c-6fb2-4ff8-80d7-d632512bf8b2 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.560962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 499/1000] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:09.563412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d1b4c-6fb2-4ff8-80d7-d632512bf8b2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:09.572223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a317f-7993-4c13-af94-3ed09aea558d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:09.607801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bf6fa-691a-4f35-9344-053e9452be38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:09.608337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bf6fa-691a-4f35-9344-053e9452be38 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.608620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bf6fa-691a-4f35-9344-053e9452be38 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.608923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bf6fa-691a-4f35-9344-053e9452be38 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.609515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537bf6fa-691a-4f35-9344-053e9452be38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:09.610889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537bf6fa-691a-4f35-9344-053e9452be38 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:09.617030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 502/1001] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Sep 15 10:44:09 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:44:09.618316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c47f7-d71a-4da1-b1de-63551ed45f6e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.618598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c47f7-d71a-4da1-b1de-63551ed45f6e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.618869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c47f7-d71a-4da1-b1de-63551ed45f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.639492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0f9c47f7-d71a-4da1-b1de-63551ed45f6e None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:09.641123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0f9c47f7-d71a-4da1-b1de-63551ed45f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:09.659739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:09.685345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d1b4c-6fb2-4ff8-80d7-d632512bf8b2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:09.726553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 503/1002] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:09.729615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b52a4-f771-4197-8a52-99c3bb3ed880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:09.730865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b52a4-f771-4197-8a52-99c3bb3ed880 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.731142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b52a4-f771-4197-8a52-99c3bb3ed880 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.731422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b52a4-f771-4197-8a52-99c3bb3ed880 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.732046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b52a4-f771-4197-8a52-99c3bb3ed880 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:09.732350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b52a4-f771-4197-8a52-99c3bb3ed880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:09.733783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b52a4-f771-4197-8a52-99c3bb3ed880 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:09.744209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 500/1003] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Sep 15 10:44:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:09.745254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c47f7-d71a-4da1-b1de-63551ed45f6e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:09.854130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c47f7-d71a-4da1-b1de-63551ed45f6e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:09.892204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79121ff1-2846-44b9-b637-a2dba4f95362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:09.893198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79121ff1-2846-44b9-b637-a2dba4f95362 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:09.893697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79121ff1-2846-44b9-b637-a2dba4f95362 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:09.894178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79121ff1-2846-44b9-b637-a2dba4f95362 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:09.895031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79121ff1-2846-44b9-b637-a2dba4f95362 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:09.896599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79121ff1-2846-44b9-b637-a2dba4f95362 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:09.915305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 504/1004] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:44:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:09.999750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b52a4-f771-4197-8a52-99c3bb3ed880 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:10.009442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:10.113674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b52a4-f771-4197-8a52-99c3bb3ed880 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:10.154881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8d169-a834-4e2f-a356-c4d6fe4769ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:10.155515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8d169-a834-4e2f-a356-c4d6fe4769ab None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:10.155797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8d169-a834-4e2f-a356-c4d6fe4769ab None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:10.156067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8d169-a834-4e2f-a356-c4d6fe4769ab None admin] PATH_INFO: `/v3/services/image` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:10.156647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8d169-a834-4e2f-a356-c4d6fe4769ab None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:10.158030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8d169-a834-4e2f-a356-c4d6fe4769ab None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-14a8d169-a834-4e2f-a356-c4d6fe4769ab None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:10.166929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:10.168980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:10.169893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 501/1005] 10.222.0.50 () {56 vars in 1105 bytes} [Thu Sep 15 10:44:09 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:10.174760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79121ff1-2846-44b9-b637-a2dba4f95362 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:10.188697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:10.282711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79121ff1-2846-44b9-b637-a2dba4f95362 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:10.317550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e94f16-ccba-4cf9-8c93-8001bd9df5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:10.318164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e94f16-ccba-4cf9-8c93-8001bd9df5ad None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:10.318450 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e94f16-ccba-4cf9-8c93-8001bd9df5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:10.318720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e94f16-ccba-4cf9-8c93-8001bd9df5ad None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:10.319343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e94f16-ccba-4cf9-8c93-8001bd9df5ad None admin] RBAC: Adding query filter params (name=image) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:10.319643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e94f16-ccba-4cf9-8c93-8001bd9df5ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:10.321052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e94f16-ccba-4cf9-8c93-8001bd9df5ad None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:10.326360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 505/1006] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:10.332184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8d169-a834-4e2f-a356-c4d6fe4769ab None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:10.401496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:10.440193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8d169-a834-4e2f-a356-c4d6fe4769ab None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:10.474871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d1ad4-a48c-407f-8197-945b31abc660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:10.475482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d1ad4-a48c-407f-8197-945b31abc660 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:10.475766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d1ad4-a48c-407f-8197-945b31abc660 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:10.476042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d1ad4-a48c-407f-8197-945b31abc660 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:10.476670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d1ad4-a48c-407f-8197-945b31abc660 None admin] RBAC: Adding query filter params (type=image) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:10.477007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d1ad4-a48c-407f-8197-945b31abc660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:10.478391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d1ad4-a48c-407f-8197-945b31abc660 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:10.483568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 502/1007] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:11.833802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc51cf2-6eb2-4846-a476-d97571c60e02 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:11.834299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc51cf2-6eb2-4846-a476-d97571c60e02 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:11.834299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc51cf2-6eb2-4846-a476-d97571c60e02 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:11.835602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 506/1008] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:11.841134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda65b1a-b788-437a-a00e-91b9da9cf52f None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:11.841560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda65b1a-b788-437a-a00e-91b9da9cf52f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:11.841677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda65b1a-b788-437a-a00e-91b9da9cf52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:11.860562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-fda65b1a-b788-437a-a00e-91b9da9cf52f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:11.861818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fda65b1a-b788-437a-a00e-91b9da9cf52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:11.936374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 503/1009] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:12.116979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e23ad2-6823-4cda-8204-3c8fa78762fc None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.117199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e23ad2-6823-4cda-8204-3c8fa78762fc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.117452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e23ad2-6823-4cda-8204-3c8fa78762fc None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.118795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 507/1010] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:12.124904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed675e33-e273-46a4-8ab3-c5f38cd79d81 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.125269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed675e33-e273-46a4-8ab3-c5f38cd79d81 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.125625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed675e33-e273-46a4-8ab3-c5f38cd79d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.144790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ed675e33-e273-46a4-8ab3-c5f38cd79d81 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:12.146044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ed675e33-e273-46a4-8ab3-c5f38cd79d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:12.204601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e23ad2-6823-4cda-8204-3c8fa78762fc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:12.205510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:12.218223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 504/1011] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:12.290727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:12.317902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e23ad2-6823-4cda-8204-3c8fa78762fc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:12.353390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cab8d9-358d-4fc7-ab5c-96514d01ba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:12.354025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cab8d9-358d-4fc7-ab5c-96514d01ba6b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.354305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cab8d9-358d-4fc7-ab5c-96514d01ba6b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.354579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cab8d9-358d-4fc7-ab5c-96514d01ba6b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.355252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cab8d9-358d-4fc7-ab5c-96514d01ba6b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:12.356676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cab8d9-358d-4fc7-ab5c-96514d01ba6b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-12cab8d9-358d-4fc7-ab5c-96514d01ba6b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:12.365649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:12.366742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:12.368242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 508/1012] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:44:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:12.372852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed675e33-e273-46a4-8ab3-c5f38cd79d81 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:12.475153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed675e33-e273-46a4-8ab3-c5f38cd79d81 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:12.489776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cab8d9-358d-4fc7-ab5c-96514d01ba6b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:12.516045 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ee7af-e8ac-473b-bdcf-79f3cf89e3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:12.516679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ee7af-e8ac-473b-bdcf-79f3cf89e3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.516988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ee7af-e8ac-473b-bdcf-79f3cf89e3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.517262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ee7af-e8ac-473b-bdcf-79f3cf89e3b5 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.517908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ee7af-e8ac-473b-bdcf-79f3cf89e3b5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:12.518213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ee7af-e8ac-473b-bdcf-79f3cf89e3b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:12.519636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ee7af-e8ac-473b-bdcf-79f3cf89e3b5 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:12.529049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 505/1013] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:44:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:12.532295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ee7af-e8ac-473b-bdcf-79f3cf89e3b5 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:12.599130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:12.610797 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cab8d9-358d-4fc7-ab5c-96514d01ba6b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:12.645917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bcb94-adac-451f-ae97-43f69c9d8373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:12.646444 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bcb94-adac-451f-ae97-43f69c9d8373 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.646719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bcb94-adac-451f-ae97-43f69c9d8373 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.646890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ee7af-e8ac-473b-bdcf-79f3cf89e3b5 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:12.646995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bcb94-adac-451f-ae97-43f69c9d8373 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.648034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bcb94-adac-451f-ae97-43f69c9d8373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:12.649499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bcb94-adac-451f-ae97-43f69c9d8373 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:12.651043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 509/1014] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:12.654706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc49022-2205-4f0d-b95f-e2b0020d65c2 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.654706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc49022-2205-4f0d-b95f-e2b0020d65c2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.654706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc49022-2205-4f0d-b95f-e2b0020d65c2 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.654706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 510/1015] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:12.657633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc49022-2205-4f0d-b95f-e2b0020d65c2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:12.666153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:12.682801 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2aa28b-42f5-474b-9fe0-01d33d9d70c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:12.683734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa28b-42f5-474b-9fe0-01d33d9d70c8 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.684017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa28b-42f5-474b-9fe0-01d33d9d70c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.684286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa28b-42f5-474b-9fe0-01d33d9d70c8 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.684939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2aa28b-42f5-474b-9fe0-01d33d9d70c8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:12.685243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2aa28b-42f5-474b-9fe0-01d33d9d70c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:12.686618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2aa28b-42f5-474b-9fe0-01d33d9d70c8 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:12.691202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 506/1016] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Sep 15 10:44:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:12.692313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfd242-f092-4f50-89e0-217943f02632 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.692600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfd242-f092-4f50-89e0-217943f02632 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.692902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfd242-f092-4f50-89e0-217943f02632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.710868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-cfcfd242-f092-4f50-89e0-217943f02632 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:12.712081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cfcfd242-f092-4f50-89e0-217943f02632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:12.759536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc49022-2205-4f0d-b95f-e2b0020d65c2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:12.782362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 507/1017] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:12.796947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfd242-f092-4f50-89e0-217943f02632 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:12.797058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639f183-ee23-4ede-aac8-1abd9238361b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:12.797058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639f183-ee23-4ede-aac8-1abd9238361b None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.797058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639f183-ee23-4ede-aac8-1abd9238361b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.797058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639f183-ee23-4ede-aac8-1abd9238361b None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.797401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639f183-ee23-4ede-aac8-1abd9238361b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:12.797401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639f183-ee23-4ede-aac8-1abd9238361b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:12.826801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 511/1018] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:44:12 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:12.842671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:12.907387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfd242-f092-4f50-89e0-217943f02632 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:12.946295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49d801-59e3-4136-869c-0ad56129e79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:12.947226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49d801-59e3-4136-869c-0ad56129e79b None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:12.947649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49d801-59e3-4136-869c-0ad56129e79b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:12.948062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49d801-59e3-4136-869c-0ad56129e79b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:12.949674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49d801-59e3-4136-869c-0ad56129e79b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:12.951701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49d801-59e3-4136-869c-0ad56129e79b None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:12.978056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 508/1019] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:44:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:13.062833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639f183-ee23-4ede-aac8-1abd9238361b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:13.129256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:13.177435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639f183-ee23-4ede-aac8-1abd9238361b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:13.229565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6572cbb-42e8-47c3-944e-08127d86d084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:13.230794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6572cbb-42e8-47c3-944e-08127d86d084 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:13.231010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6572cbb-42e8-47c3-944e-08127d86d084 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:13.231346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6572cbb-42e8-47c3-944e-08127d86d084 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:13.232025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6572cbb-42e8-47c3-944e-08127d86d084 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:13.233428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6572cbb-42e8-47c3-944e-08127d86d084 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:13.249561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 512/1020] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:44:13 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:15.055605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e7459-b293-4c75-b8e1-6e981d30361d None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.056157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e7459-b293-4c75-b8e1-6e981d30361d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.056157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e7459-b293-4c75-b8e1-6e981d30361d None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.057701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 509/1021] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:15.063276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc2137-4407-4b07-be4f-02eaa8ef1b8d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.063669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc2137-4407-4b07-be4f-02eaa8ef1b8d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.063859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc2137-4407-4b07-be4f-02eaa8ef1b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.082988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b7fc2137-4407-4b07-be4f-02eaa8ef1b8d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:15.084143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b7fc2137-4407-4b07-be4f-02eaa8ef1b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:15.162257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 513/1022] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:15.271187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28c40c-7325-4625-8c70-3bf3e51d6cc8 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.271555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28c40c-7325-4625-8c70-3bf3e51d6cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.271732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28c40c-7325-4625-8c70-3bf3e51d6cc8 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.273591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 510/1023] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:15.278605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceecbf-a6cc-4c18-b5c2-90cc70b60734 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.278904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceecbf-a6cc-4c18-b5c2-90cc70b60734 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.279184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceecbf-a6cc-4c18-b5c2-90cc70b60734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.299064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8fceecbf-a6cc-4c18-b5c2-90cc70b60734 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:15.300314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8fceecbf-a6cc-4c18-b5c2-90cc70b60734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:15.338673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:15.375834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 514/1024] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:15.414577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021fcd9-c955-4043-bf3c-ba923f231da4 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.414926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021fcd9-c955-4043-bf3c-ba923f231da4 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.415134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021fcd9-c955-4043-bf3c-ba923f231da4 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.416501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 511/1025] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:15.422320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd6524-99fb-4c5f-b513-5bab3a372495 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.422607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd6524-99fb-4c5f-b513-5bab3a372495 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.422875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd6524-99fb-4c5f-b513-5bab3a372495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.434746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021fcd9-c955-4043-bf3c-ba923f231da4 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:15.441653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6ebd6524-99fb-4c5f-b513-5bab3a372495 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:15.442979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6ebd6524-99fb-4c5f-b513-5bab3a372495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:15.514120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 515/1026] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:15.548614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021fcd9-c955-4043-bf3c-ba923f231da4 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:15.551806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:15.589491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb4db0-e635-4dbf-b6ff-731ba58e7395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:15.590048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb4db0-e635-4dbf-b6ff-731ba58e7395 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.590329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb4db0-e635-4dbf-b6ff-731ba58e7395 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.590604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb4db0-e635-4dbf-b6ff-731ba58e7395 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.591743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb4db0-e635-4dbf-b6ff-731ba58e7395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:15.593246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb4db0-e635-4dbf-b6ff-731ba58e7395 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:15.594980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 512/1027] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:15.600438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd6524-99fb-4c5f-b513-5bab3a372495 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:15.644645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb4db0-e635-4dbf-b6ff-731ba58e7395 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:15.705600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:15.709157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd6524-99fb-4c5f-b513-5bab3a372495 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:15.745360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561a942-f1c8-4535-a111-78ae35b0374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:15.746069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561a942-f1c8-4535-a111-78ae35b0374c None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.746357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561a942-f1c8-4535-a111-78ae35b0374c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.746630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561a942-f1c8-4535-a111-78ae35b0374c None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.747277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561a942-f1c8-4535-a111-78ae35b0374c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:15.748720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561a942-f1c8-4535-a111-78ae35b0374c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:15.756745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb4db0-e635-4dbf-b6ff-731ba58e7395 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:15.771737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 516/1028] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:44:15 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:15.787370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561a942-f1c8-4535-a111-78ae35b0374c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:15.799605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ad4a4-a38b-4052-b94e-ead181f4ca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:15.800267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad4a4-a38b-4052-b94e-ead181f4ca7d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.800551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad4a4-a38b-4052-b94e-ead181f4ca7d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.800979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad4a4-a38b-4052-b94e-ead181f4ca7d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.814955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ad4a4-a38b-4052-b94e-ead181f4ca7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-922ad4a4-a38b-4052-b94e-ead181f4ca7d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:15.819971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:15.821171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:15.822066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 513/1029] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Sep 15 10:44:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:15.827626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ad4a4-a38b-4052-b94e-ead181f4ca7d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:15.863257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:15.909496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561a942-f1c8-4535-a111-78ae35b0374c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:15.945387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ad4a4-a38b-4052-b94e-ead181f4ca7d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:15.949429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210bf35-f248-4639-8b54-52d9cd0fcb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:15.951116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210bf35-f248-4639-8b54-52d9cd0fcb7b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.951116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210bf35-f248-4639-8b54-52d9cd0fcb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.951116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210bf35-f248-4639-8b54-52d9cd0fcb7b None admin] PATH_INFO: `/v3/services/image` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.951472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210bf35-f248-4639-8b54-52d9cd0fcb7b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:15.953048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210bf35-f248-4639-8b54-52d9cd0fcb7b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-f210bf35-f248-4639-8b54-52d9cd0fcb7b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:15.962096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:15.963127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:15.964551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 517/1030] 10.222.0.50 () {56 vars in 1105 bytes} [Thu Sep 15 10:44:15 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:15.970438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210bf35-f248-4639-8b54-52d9cd0fcb7b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:15.991361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1572e-e2a2-45e3-a4df-7bb02c8fc009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:15.992333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1572e-e2a2-45e3-a4df-7bb02c8fc009 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:15.992333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1572e-e2a2-45e3-a4df-7bb02c8fc009 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:15.992529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1572e-e2a2-45e3-a4df-7bb02c8fc009 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:15.993186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1572e-e2a2-45e3-a4df-7bb02c8fc009 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:15.994005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1572e-e2a2-45e3-a4df-7bb02c8fc009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:15.994777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:15.994897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1572e-e2a2-45e3-a4df-7bb02c8fc009 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:16.006431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 514/1031] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Sep 15 10:44:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:16.010356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1572e-e2a2-45e3-a4df-7bb02c8fc009 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:16.090326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210bf35-f248-4639-8b54-52d9cd0fcb7b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:16.113561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:16.122599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1572e-e2a2-45e3-a4df-7bb02c8fc009 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:16.132966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1155b67-ff1a-4d21-a7a3-9e6796e369af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:16.133891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1155b67-ff1a-4d21-a7a3-9e6796e369af None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:16.134320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1155b67-ff1a-4d21-a7a3-9e6796e369af None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:16.134594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1155b67-ff1a-4d21-a7a3-9e6796e369af None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:16.135221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1155b67-ff1a-4d21-a7a3-9e6796e369af None admin] RBAC: Adding query filter params (name=image) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:16.135630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1155b67-ff1a-4d21-a7a3-9e6796e369af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:16.137157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1155b67-ff1a-4d21-a7a3-9e6796e369af None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:16.142108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 518/1032] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:16.148688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1155b67-ff1a-4d21-a7a3-9e6796e369af None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:16.162992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5831b-2057-47a6-83ec-822698091ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:16.163628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5831b-2057-47a6-83ec-822698091ebc None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:16.163902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5831b-2057-47a6-83ec-822698091ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:16.164168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5831b-2057-47a6-83ec-822698091ebc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:16.175741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5831b-2057-47a6-83ec-822698091ebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:16.177353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5831b-2057-47a6-83ec-822698091ebc None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-84a5831b-2057-47a6-83ec-822698091ebc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:16.193375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:16.194614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:16.195899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:16.195899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:16.195899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:16.200271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 515/1033] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:16.201178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5831b-2057-47a6-83ec-822698091ebc None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:16.244520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:16.262745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1155b67-ff1a-4d21-a7a3-9e6796e369af None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:16.299097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5909-6388-45b3-9c7c-2c8d65582cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:16.299708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5909-6388-45b3-9c7c-2c8d65582cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:16.299991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5909-6388-45b3-9c7c-2c8d65582cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:16.300254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5909-6388-45b3-9c7c-2c8d65582cd0 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:16.300968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5909-6388-45b3-9c7c-2c8d65582cd0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:16.301268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5909-6388-45b3-9c7c-2c8d65582cd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:16.302740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5909-6388-45b3-9c7c-2c8d65582cd0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:16.309053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5831b-2057-47a6-83ec-822698091ebc None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:16.312849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 519/1034] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:16.314919 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5909-6388-45b3-9c7c-2c8d65582cd0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:16.346204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b62d3-6b49-4edb-9273-bd3b18185d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:16.346819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b62d3-6b49-4edb-9273-bd3b18185d0e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:16.347105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b62d3-6b49-4edb-9273-bd3b18185d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:16.347382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b62d3-6b49-4edb-9273-bd3b18185d0e None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:16.348029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b62d3-6b49-4edb-9273-bd3b18185d0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:16.348334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b62d3-6b49-4edb-9273-bd3b18185d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:16.349743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b62d3-6b49-4edb-9273-bd3b18185d0e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:16.362415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 516/1035] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:16.368298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b62d3-6b49-4edb-9273-bd3b18185d0e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:16.378986 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:16.421447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5909-6388-45b3-9c7c-2c8d65582cd0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:16.456934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74e0c5-454b-44d9-90a2-0a938b7fa15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:16.457544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74e0c5-454b-44d9-90a2-0a938b7fa15b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:16.457827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74e0c5-454b-44d9-90a2-0a938b7fa15b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:16.458095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74e0c5-454b-44d9-90a2-0a938b7fa15b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:16.458784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74e0c5-454b-44d9-90a2-0a938b7fa15b None admin] RBAC: Adding query filter params (interface=public, service_id=4b68377b7ca24963aac570c38d0f5338, region_id=RegionOne) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:16.459095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74e0c5-454b-44d9-90a2-0a938b7fa15b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:16.460463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74e0c5-454b-44d9-90a2-0a938b7fa15b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:16.465447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 520/1036] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Sep 15 10:44:16 2022] GET /identity/v3/endpoints?service_id=4b68377b7ca24963aac570c38d0f5338&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:16.471029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74e0c5-454b-44d9-90a2-0a938b7fa15b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:16.482713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b62d3-6b49-4edb-9273-bd3b18185d0e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:16.486098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:16.518811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ccc9e-2e4f-4395-a9c1-4db9ebebb8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:16.519440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ccc9e-2e4f-4395-a9c1-4db9ebebb8ef None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:16.519717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ccc9e-2e4f-4395-a9c1-4db9ebebb8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:16.519995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ccc9e-2e4f-4395-a9c1-4db9ebebb8ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:16.527544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ccc9e-2e4f-4395-a9c1-4db9ebebb8ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:16.529014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ccc9e-2e4f-4395-a9c1-4db9ebebb8ef None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-120ccc9e-2e4f-4395-a9c1-4db9ebebb8ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:44:16.540839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:16.541836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:16.543252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 517/1037] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Sep 15 10:44:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:16.547716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ccc9e-2e4f-4395-a9c1-4db9ebebb8ef None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:16.593845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74e0c5-454b-44d9-90a2-0a938b7fa15b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:16.615563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:16.630573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6162e83-a19d-477a-83de-9852fe717f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:16.631221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6162e83-a19d-477a-83de-9852fe717f04 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:16.631504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6162e83-a19d-477a-83de-9852fe717f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:16.631770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6162e83-a19d-477a-83de-9852fe717f04 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:16.632367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6162e83-a19d-477a-83de-9852fe717f04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:16.634001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6162e83-a19d-477a-83de-9852fe717f04 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:16.648879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 521/1038] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Sep 15 10:44:16 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:44:16.662720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ccc9e-2e4f-4395-a9c1-4db9ebebb8ef None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:16.702329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b4056-0f65-4e2c-a540-eeb923b1c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:16.703000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b4056-0f65-4e2c-a540-eeb923b1c61a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:16.703284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b4056-0f65-4e2c-a540-eeb923b1c61a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:16.703557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b4056-0f65-4e2c-a540-eeb923b1c61a None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:16.704315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b4056-0f65-4e2c-a540-eeb923b1c61a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:16.704622 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b4056-0f65-4e2c-a540-eeb923b1c61a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:16.706130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b4056-0f65-4e2c-a540-eeb923b1c61a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:16.727765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 518/1039] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Sep 15 10:44:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:16.734179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6162e83-a19d-477a-83de-9852fe717f04 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:16.818040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:16.845101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6162e83-a19d-477a-83de-9852fe717f04 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:16.880107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518bcc8-2cb5-45d6-b026-358ffdd35076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:16.881084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518bcc8-2cb5-45d6-b026-358ffdd35076 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:16.881548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518bcc8-2cb5-45d6-b026-358ffdd35076 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:16.881998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518bcc8-2cb5-45d6-b026-358ffdd35076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:16.954320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518bcc8-2cb5-45d6-b026-358ffdd35076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:16.956060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518bcc8-2cb5-45d6-b026-358ffdd35076 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:17.036915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:17.063774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 522/1040] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:16 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:44:17.069529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b4056-0f65-4e2c-a540-eeb923b1c61a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:17.174263 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b4056-0f65-4e2c-a540-eeb923b1c61a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:17.197666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:17.211492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a209db7-3ff2-4209-acc1-86cab18fc666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:17.212134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a209db7-3ff2-4209-acc1-86cab18fc666 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:17.212417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a209db7-3ff2-4209-acc1-86cab18fc666 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:17.212690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a209db7-3ff2-4209-acc1-86cab18fc666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:17.223941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a209db7-3ff2-4209-acc1-86cab18fc666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:17.225431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a209db7-3ff2-4209-acc1-86cab18fc666 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-7a209db7-3ff2-4209-acc1-86cab18fc666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:17.240552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:17.241576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:17.242490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:17.242490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:17.242490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:17.245127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 519/1041] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:17.248046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518bcc8-2cb5-45d6-b026-358ffdd35076 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:17.349918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518bcc8-2cb5-45d6-b026-358ffdd35076 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:17.384780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75516c75-b606-466c-868f-4c5ef36f5e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:17.385425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516c75-b606-466c-868f-4c5ef36f5e4e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:17.385709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516c75-b606-466c-868f-4c5ef36f5e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:17.386530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516c75-b606-466c-868f-4c5ef36f5e4e None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:17.387318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75516c75-b606-466c-868f-4c5ef36f5e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:17.387625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75516c75-b606-466c-868f-4c5ef36f5e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:17.389036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75516c75-b606-466c-868f-4c5ef36f5e4e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:17.405820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 523/1042] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:17.407696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a209db7-3ff2-4209-acc1-86cab18fc666 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:17.512565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a209db7-3ff2-4209-acc1-86cab18fc666 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:17.556751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f444-c1bc-4a92-ab29-053d4a15c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:17.557429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f444-c1bc-4a92-ab29-053d4a15c3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:17.558332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f444-c1bc-4a92-ab29-053d4a15c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:17.558619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f444-c1bc-4a92-ab29-053d4a15c3a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:17.566614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93f444-c1bc-4a92-ab29-053d4a15c3a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:17.568067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93f444-c1bc-4a92-ab29-053d4a15c3a6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-aa93f444-c1bc-4a92-ab29-053d4a15c3a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:44:17.580531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:17.581456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:17.582390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:44:17.582390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:44:17.582390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:44:17.582390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:44:17.582390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:44:17.582390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:44:17.582390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:17.582390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:17.583321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 520/1043] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:44:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:17.587786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75516c75-b606-466c-868f-4c5ef36f5e4e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:17.625215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:17.693263 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75516c75-b606-466c-868f-4c5ef36f5e4e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:17.728169 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e0724-6f4a-4662-ad4c-c075d5857fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:17.728780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e0724-6f4a-4662-ad4c-c075d5857fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:17.729099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e0724-6f4a-4662-ad4c-c075d5857fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:17.729370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e0724-6f4a-4662-ad4c-c075d5857fe4 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:17.730261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e0724-6f4a-4662-ad4c-c075d5857fe4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:17.730573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e0724-6f4a-4662-ad4c-c075d5857fe4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:17.732014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e0724-6f4a-4662-ad4c-c075d5857fe4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:17.744210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 524/1044] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:44:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:17.750221 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f444-c1bc-4a92-ab29-053d4a15c3a6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:17.787913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:17.855105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f444-c1bc-4a92-ab29-053d4a15c3a6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:17.891254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2e503-dc1c-4603-866d-1b0c5d2021ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:17.891890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e503-dc1c-4603-866d-1b0c5d2021ff None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:17.892169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e503-dc1c-4603-866d-1b0c5d2021ff None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:17.892440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e503-dc1c-4603-866d-1b0c5d2021ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:17.893187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2e503-dc1c-4603-866d-1b0c5d2021ff None admin] RBAC: Adding query filter params (role.id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=218a015dd2b34435a76ecc8a763562ed) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:17.893494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2e503-dc1c-4603-866d-1b0c5d2021ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:17.894954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2e503-dc1c-4603-866d-1b0c5d2021ff None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:17.901330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 521/1045] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:44:17 2022] GET /identity/v3/role_assignments?user.id=218a015dd2b34435a76ecc8a763562ed&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:18.029754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63063af-8369-4037-a16c-43fabbaa9ebb None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:18.030118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63063af-8369-4037-a16c-43fabbaa9ebb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:18.030308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63063af-8369-4037-a16c-43fabbaa9ebb None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:18.031860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 525/1046] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:18.037386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0943d-1a25-46a4-902c-d01d140b55fd None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:18.037761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0943d-1a25-46a4-902c-d01d140b55fd None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:18.038166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0943d-1a25-46a4-902c-d01d140b55fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:18.058126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-84f0943d-1a25-46a4-902c-d01d140b55fd None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:18.059353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-84f0943d-1a25-46a4-902c-d01d140b55fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:18.132847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 522/1047] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:18.422913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63063af-8369-4037-a16c-43fabbaa9ebb None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:18.526672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:18.533550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63063af-8369-4037-a16c-43fabbaa9ebb None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:18.571897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f948412-4b4a-40d4-8b42-c2c642bd8fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:18.572538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f948412-4b4a-40d4-8b42-c2c642bd8fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:18.572891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f948412-4b4a-40d4-8b42-c2c642bd8fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:18.573137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f948412-4b4a-40d4-8b42-c2c642bd8fd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:18.584952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f948412-4b4a-40d4-8b42-c2c642bd8fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2f948412-4b4a-40d4-8b42-c2c642bd8fd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:18.590394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:18.591337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:18.597400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 526/1048] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:44:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:18.599220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0943d-1a25-46a4-902c-d01d140b55fd None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:18.638399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:18.712951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0943d-1a25-46a4-902c-d01d140b55fd None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:18.748311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90221c8-8660-4bee-adeb-2376c6f15650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:18.748979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90221c8-8660-4bee-adeb-2376c6f15650 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:18.749263 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90221c8-8660-4bee-adeb-2376c6f15650 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:18.749533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90221c8-8660-4bee-adeb-2376c6f15650 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:18.750219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90221c8-8660-4bee-adeb-2376c6f15650 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:18.750522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90221c8-8660-4bee-adeb-2376c6f15650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:18.751940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90221c8-8660-4bee-adeb-2376c6f15650 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:18.767054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 523/1049] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:44:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:18.768289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f948412-4b4a-40d4-8b42-c2c642bd8fd4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:18.867065 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da5271-cd35-40ab-92cf-1d46146c0660 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:18.867397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da5271-cd35-40ab-92cf-1d46146c0660 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:18.867604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da5271-cd35-40ab-92cf-1d46146c0660 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:18.869228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 524/1050] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:44:18.874450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520d12a-3de2-4ee5-b2f2-1cb2dbec3f6a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:18.874575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f948412-4b4a-40d4-8b42-c2c642bd8fd4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:18.874779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520d12a-3de2-4ee5-b2f2-1cb2dbec3f6a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:18.874997 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520d12a-3de2-4ee5-b2f2-1cb2dbec3f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:18.894188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-6520d12a-3de2-4ee5-b2f2-1cb2dbec3f6a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:18.895484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6520d12a-3de2-4ee5-b2f2-1cb2dbec3f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:18.912201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c90911-f59b-4c21-aa81-1e680d13e3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:18.912854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c90911-f59b-4c21-aa81-1e680d13e3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:18.913139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c90911-f59b-4c21-aa81-1e680d13e3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:18.913419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c90911-f59b-4c21-aa81-1e680d13e3c9 None admin] PATH_INFO: `/v3/users/18bf9d7dfbf34d64a11b5d6101c28a8d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:18.928302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c90911-f59b-4c21-aa81-1e680d13e3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=18bf9d7dfbf34d64a11b5d6101c28a8d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:18.929854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c90911-f59b-4c21-aa81-1e680d13e3c9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:18.931782 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 527/1051] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:18 2022] GET /identity/v3/users/18bf9d7dfbf34d64a11b5d6101c28a8d => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:18.937393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c90911-f59b-4c21-aa81-1e680d13e3c9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:18.952774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:18.970197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 525/1052] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:19.041466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c90911-f59b-4c21-aa81-1e680d13e3c9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:19.076965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4e47b-13d2-4aff-b380-0773251d8143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:19.077579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4e47b-13d2-4aff-b380-0773251d8143 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:19.077860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4e47b-13d2-4aff-b380-0773251d8143 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:19.078131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4e47b-13d2-4aff-b380-0773251d8143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:19.148680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4e47b-13d2-4aff-b380-0773251d8143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:19.150172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4e47b-13d2-4aff-b380-0773251d8143 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:19.165361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:19.239769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520d12a-3de2-4ee5-b2f2-1cb2dbec3f6a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:19.266616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:19.270939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 528/1053] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:18 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 10:44:19.277180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4e47b-13d2-4aff-b380-0773251d8143 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:19.351666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520d12a-3de2-4ee5-b2f2-1cb2dbec3f6a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:19.382625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4e47b-13d2-4aff-b380-0773251d8143 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:19.386805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3755cce-18b5-4ea3-a57f-bdcab9815979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:19.387423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3755cce-18b5-4ea3-a57f-bdcab9815979 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:19.387700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3755cce-18b5-4ea3-a57f-bdcab9815979 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:19.387970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3755cce-18b5-4ea3-a57f-bdcab9815979 None admin] PATH_INFO: `/v3/services/image` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:19.388549 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3755cce-18b5-4ea3-a57f-bdcab9815979 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:19.389952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3755cce-18b5-4ea3-a57f-bdcab9815979 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:19.398280 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-c3755cce-18b5-4ea3-a57f-bdcab9815979 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:19.398742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:19.399826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:19.400764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 526/1054] 10.222.0.50 () {56 vars in 1105 bytes} [Thu Sep 15 10:44:19 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:19.406057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3755cce-18b5-4ea3-a57f-bdcab9815979 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:19.423355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d07a8-5f8c-4ee6-a755-ef7911b4f9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:19.424667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d07a8-5f8c-4ee6-a755-ef7911b4f9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:19.425130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d07a8-5f8c-4ee6-a755-ef7911b4f9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:19.425561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d07a8-5f8c-4ee6-a755-ef7911b4f9c5 None admin] PATH_INFO: `/v3/projects/b240c81156184288a3f1f540836f195b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:19.426812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d07a8-5f8c-4ee6-a755-ef7911b4f9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=b240c81156184288a3f1f540836f195b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:19.428465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d07a8-5f8c-4ee6-a755-ef7911b4f9c5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:19.430812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 529/1055] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:44:19 2022] GET /identity/v3/projects/b240c81156184288a3f1f540836f195b => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:19.437355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d07a8-5f8c-4ee6-a755-ef7911b4f9c5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:19.500441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:19.519025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3755cce-18b5-4ea3-a57f-bdcab9815979 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:19.540402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d07a8-5f8c-4ee6-a755-ef7911b4f9c5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:19.561549 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7298-e694-4ede-9453-16f83e7aaeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:19.562161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7298-e694-4ede-9453-16f83e7aaeaf None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:19.562439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7298-e694-4ede-9453-16f83e7aaeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:19.562707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7298-e694-4ede-9453-16f83e7aaeaf None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:19.563333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7298-e694-4ede-9453-16f83e7aaeaf None admin] RBAC: Adding query filter params (name=image) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:19.563811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7298-e694-4ede-9453-16f83e7aaeaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:19.565648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7298-e694-4ede-9453-16f83e7aaeaf None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:19.570705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 527/1056] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:19.576974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7298-e694-4ede-9453-16f83e7aaeaf None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:19.586376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ca25b-751c-4256-a021-ce070f9e2f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:19.586983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ca25b-751c-4256-a021-ce070f9e2f02 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:19.587262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ca25b-751c-4256-a021-ce070f9e2f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:19.587529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ca25b-751c-4256-a021-ce070f9e2f02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:19.588240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ca25b-751c-4256-a021-ce070f9e2f02 None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=b240c81156184288a3f1f540836f195b, user.id=18bf9d7dfbf34d64a11b5d6101c28a8d) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:19.588546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ca25b-751c-4256-a021-ce070f9e2f02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:19.590374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ca25b-751c-4256-a021-ce070f9e2f02 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:19.596478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 530/1057] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:44:19 2022] GET /identity/v3/role_assignments?user.id=18bf9d7dfbf34d64a11b5d6101c28a8d&scope.project.id=b240c81156184288a3f1f540836f195b&role.id=0be9777002c641b4a47f78df647cc173 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:19.689725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7298-e694-4ede-9453-16f83e7aaeaf None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:19.727084 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c721a-1dd7-4268-8bd0-bdead7260a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:19.727709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c721a-1dd7-4268-8bd0-bdead7260a5a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:19.727983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c721a-1dd7-4268-8bd0-bdead7260a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:19.728250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c721a-1dd7-4268-8bd0-bdead7260a5a None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:19.728921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c721a-1dd7-4268-8bd0-bdead7260a5a None admin] RBAC: Adding query filter params (type=image) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:19.729220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c721a-1dd7-4268-8bd0-bdead7260a5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:19.730799 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c721a-1dd7-4268-8bd0-bdead7260a5a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:19.741344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 528/1058] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:19.742467 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ca25b-751c-4256-a021-ce070f9e2f02 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:19.794127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:19.851653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ca25b-751c-4256-a021-ce070f9e2f02 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:19.886472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6947be-ef99-47a7-9f5b-8e52cecde09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:19.887278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6947be-ef99-47a7-9f5b-8e52cecde09c None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:19.887558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6947be-ef99-47a7-9f5b-8e52cecde09c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:19.887834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6947be-ef99-47a7-9f5b-8e52cecde09c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:19.888563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6947be-ef99-47a7-9f5b-8e52cecde09c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:19.889987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6947be-ef99-47a7-9f5b-8e52cecde09c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:19.916837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 531/1059] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:44:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:20.118993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024d283-77b8-4fac-92d4-168e6bdc9299 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:20.119630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024d283-77b8-4fac-92d4-168e6bdc9299 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:20.120720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024d283-77b8-4fac-92d4-168e6bdc9299 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:20.122611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 529/1060] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:20.127717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b218a55-4694-4d7e-b8d8-23a10040f1c1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:20.128013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b218a55-4694-4d7e-b8d8-23a10040f1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:20.128291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b218a55-4694-4d7e-b8d8-23a10040f1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:20.147203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5b218a55-4694-4d7e-b8d8-23a10040f1c1 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:20.148385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5b218a55-4694-4d7e-b8d8-23a10040f1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:20.224426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 532/1061] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:20.492220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024d283-77b8-4fac-92d4-168e6bdc9299 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:20.509387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:20.602186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024d283-77b8-4fac-92d4-168e6bdc9299 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:20.638284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55891d-bc3e-4e85-b59f-0dddb20a2f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:20.638916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55891d-bc3e-4e85-b59f-0dddb20a2f75 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:20.639189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55891d-bc3e-4e85-b59f-0dddb20a2f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:20.639458 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55891d-bc3e-4e85-b59f-0dddb20a2f75 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:20.650161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55891d-bc3e-4e85-b59f-0dddb20a2f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-fb55891d-bc3e-4e85-b59f-0dddb20a2f75 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:20.655617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:20.656593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:20.661843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 530/1062] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Sep 15 10:44:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:20.663471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b218a55-4694-4d7e-b8d8-23a10040f1c1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:20.692105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:20.769343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b218a55-4694-4d7e-b8d8-23a10040f1c1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:20.803950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a280335-be84-411f-b05d-cc36b1ad94b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:20.804566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280335-be84-411f-b05d-cc36b1ad94b9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:20.804870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280335-be84-411f-b05d-cc36b1ad94b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:20.805189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280335-be84-411f-b05d-cc36b1ad94b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:20.805902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a280335-be84-411f-b05d-cc36b1ad94b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:20.806208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a280335-be84-411f-b05d-cc36b1ad94b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:20.808216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a280335-be84-411f-b05d-cc36b1ad94b9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:20.817006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 533/1063] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Sep 15 10:44:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:20.823639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55891d-bc3e-4e85-b59f-0dddb20a2f75 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:20.903421 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:20.931967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55891d-bc3e-4e85-b59f-0dddb20a2f75 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:20.965608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a947a-747f-482f-a1e4-85c2cdf8e242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:20.966238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a947a-747f-482f-a1e4-85c2cdf8e242 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:20.966517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a947a-747f-482f-a1e4-85c2cdf8e242 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:20.966788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a947a-747f-482f-a1e4-85c2cdf8e242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:20.977138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a947a-747f-482f-a1e4-85c2cdf8e242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:20.978591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a947a-747f-482f-a1e4-85c2cdf8e242 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-6b0a947a-747f-482f-a1e4-85c2cdf8e242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:20.993548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:20.994551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:20.995977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:20.995977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:20.995977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:20.996885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 531/1064] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:21.000416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a280335-be84-411f-b05d-cc36b1ad94b9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:21.092476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:21.103506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a280335-be84-411f-b05d-cc36b1ad94b9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:21.137613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25077b6-e439-4c65-887a-4622b9da0bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:21.138224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25077b6-e439-4c65-887a-4622b9da0bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:21.138507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25077b6-e439-4c65-887a-4622b9da0bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:21.138777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25077b6-e439-4c65-887a-4622b9da0bc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:21.139412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25077b6-e439-4c65-887a-4622b9da0bc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:21.139716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25077b6-e439-4c65-887a-4622b9da0bc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:21.141156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25077b6-e439-4c65-887a-4622b9da0bc3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:21.153752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 534/1065] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:21.159194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a947a-747f-482f-a1e4-85c2cdf8e242 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:21.262754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a947a-747f-482f-a1e4-85c2cdf8e242 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:21.287879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:21.298152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012429d3-f5d3-4bb9-b2a7-cd73a3de28ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:21.298793 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012429d3-f5d3-4bb9-b2a7-cd73a3de28ae None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:21.299074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012429d3-f5d3-4bb9-b2a7-cd73a3de28ae None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:21.299345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012429d3-f5d3-4bb9-b2a7-cd73a3de28ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:21.306974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012429d3-f5d3-4bb9-b2a7-cd73a3de28ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:21.309028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012429d3-f5d3-4bb9-b2a7-cd73a3de28ae None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-012429d3-f5d3-4bb9-b2a7-cd73a3de28ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:44:21.320248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:21.321437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:44:21.322418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:21.325118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 532/1066] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Sep 15 10:44:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:21.327523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25077b6-e439-4c65-887a-4622b9da0bc3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:21.427621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25077b6-e439-4c65-887a-4622b9da0bc3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:21.462325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cdde5-a2d0-4517-bd61-9ed062155a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:21.462937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cdde5-a2d0-4517-bd61-9ed062155a13 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:21.463219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cdde5-a2d0-4517-bd61-9ed062155a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:21.463488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cdde5-a2d0-4517-bd61-9ed062155a13 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:21.464479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cdde5-a2d0-4517-bd61-9ed062155a13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:21.464788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cdde5-a2d0-4517-bd61-9ed062155a13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:21.466254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cdde5-a2d0-4517-bd61-9ed062155a13 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:21.483900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 535/1067] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Sep 15 10:44:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:21.489462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012429d3-f5d3-4bb9-b2a7-cd73a3de28ae None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:21.602691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012429d3-f5d3-4bb9-b2a7-cd73a3de28ae None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:21.637299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d443eed-4922-4d45-a37b-edc97893a67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:21.637937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d443eed-4922-4d45-a37b-edc97893a67f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:21.638216 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d443eed-4922-4d45-a37b-edc97893a67f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:21.638487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d443eed-4922-4d45-a37b-edc97893a67f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:21.649224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d443eed-4922-4d45-a37b-edc97893a67f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:21.650687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d443eed-4922-4d45-a37b-edc97893a67f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-1d443eed-4922-4d45-a37b-edc97893a67f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:21.666439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:21.668100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:21.669094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:21.669094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:21.669094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:21.672899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 533/1068] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:21.674212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cdde5-a2d0-4517-bd61-9ed062155a13 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:21.686604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:21.779700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cdde5-a2d0-4517-bd61-9ed062155a13 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:21.784072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dab1d-d6ea-43fe-8c0c-051fd58925b9 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:21.784354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dab1d-d6ea-43fe-8c0c-051fd58925b9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:21.784627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dab1d-d6ea-43fe-8c0c-051fd58925b9 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:21.786134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 534/1069] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:44:21.790855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6999cd-5b05-4c78-8881-4ae981324692 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:21.791144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6999cd-5b05-4c78-8881-4ae981324692 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:21.791415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6999cd-5b05-4c78-8881-4ae981324692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:21.810821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-db6999cd-5b05-4c78-8881-4ae981324692 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:21.812004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-db6999cd-5b05-4c78-8881-4ae981324692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:21.818054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e62055-7425-4584-92b7-adcfda1f1bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:21.818668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e62055-7425-4584-92b7-adcfda1f1bee None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:21.818955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e62055-7425-4584-92b7-adcfda1f1bee None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:21.819543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e62055-7425-4584-92b7-adcfda1f1bee None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:21.819858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e62055-7425-4584-92b7-adcfda1f1bee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:21.820162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e62055-7425-4584-92b7-adcfda1f1bee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:21.821566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e62055-7425-4584-92b7-adcfda1f1bee None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:21.835093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 536/1070] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:21.840777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e62055-7425-4584-92b7-adcfda1f1bee None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:21.850166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:21.893653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 535/1071] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:44:21.916538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:21.952929 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e62055-7425-4584-92b7-adcfda1f1bee None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:21.988468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6344e3-06d3-403e-9d0e-31200239bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:21.989146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6344e3-06d3-403e-9d0e-31200239bf0d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:21.989427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6344e3-06d3-403e-9d0e-31200239bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:21.989701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6344e3-06d3-403e-9d0e-31200239bf0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:21.997978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6344e3-06d3-403e-9d0e-31200239bf0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:21.999488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6344e3-06d3-403e-9d0e-31200239bf0d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-bd6344e3-06d3-403e-9d0e-31200239bf0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:44:22.013483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:22.014495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:22.015567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:44:22.015567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:44:22.015567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:44:22.015567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:44:22.015567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:44:22.015567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:44:22.015567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:22.015567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:22.017197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 537/1072] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:44:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:22.020665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6999cd-5b05-4c78-8881-4ae981324692 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:22.118973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c162d67-9585-4668-8c08-97bec9728c03 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:22.119329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c162d67-9585-4668-8c08-97bec9728c03 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:22.119534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c162d67-9585-4668-8c08-97bec9728c03 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:22.121098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 538/1073] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:44:22.126041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf7bb-04b2-4311-875f-c35f45aaff26 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:22.126311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf7bb-04b2-4311-875f-c35f45aaff26 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:22.126584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf7bb-04b2-4311-875f-c35f45aaff26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:22.127880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6999cd-5b05-4c78-8881-4ae981324692 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:22.145672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b64cf7bb-04b2-4311-875f-c35f45aaff26 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:22.146876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b64cf7bb-04b2-4311-875f-c35f45aaff26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:22.163609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec133be0-cd68-4473-869e-3e810e03f818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:22.164220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec133be0-cd68-4473-869e-3e810e03f818 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:22.164504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec133be0-cd68-4473-869e-3e810e03f818 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:22.164775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec133be0-cd68-4473-869e-3e810e03f818 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:22.165490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec133be0-cd68-4473-869e-3e810e03f818 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:22.165797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec133be0-cd68-4473-869e-3e810e03f818 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:22.167913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec133be0-cd68-4473-869e-3e810e03f818 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:22.184697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 536/1074] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:44:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:22.184697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec133be0-cd68-4473-869e-3e810e03f818 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:22.209836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:22.224238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 539/1075] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:44:22.225329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cf7bb-04b2-4311-875f-c35f45aaff26 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:22.284549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:22.293004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec133be0-cd68-4473-869e-3e810e03f818 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:22.326299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cf7bb-04b2-4311-875f-c35f45aaff26 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:22.329224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebbdb6-661a-4f70-b67e-cbf6d35b3bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:22.329847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebbdb6-661a-4f70-b67e-cbf6d35b3bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:22.330122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebbdb6-661a-4f70-b67e-cbf6d35b3bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:22.330387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebbdb6-661a-4f70-b67e-cbf6d35b3bf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:22.342406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebbdb6-661a-4f70-b67e-cbf6d35b3bf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-29ebbdb6-661a-4f70-b67e-cbf6d35b3bf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:22.347608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:22.348568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:22.349500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 537/1076] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:44:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:22.354783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebbdb6-661a-4f70-b67e-cbf6d35b3bf4 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:22.363515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48d519-f8a5-42c5-8bbe-0fe291bc2338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:22.364181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d519-f8a5-42c5-8bbe-0fe291bc2338 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:22.364462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d519-f8a5-42c5-8bbe-0fe291bc2338 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:22.364733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d519-f8a5-42c5-8bbe-0fe291bc2338 None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/218a015dd2b34435a76ecc8a763562ed/roles/b4c8a5ff4e0c44e1a0bd3beaff37e4b4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:22.366910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48d519-f8a5-42c5-8bbe-0fe291bc2338 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=218a015dd2b34435a76ecc8a763562ed, role_id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:22.368350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48d519-f8a5-42c5-8bbe-0fe291bc2338 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:22.382856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 540/1077] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:44:22 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/218a015dd2b34435a76ecc8a763562ed/roles/b4c8a5ff4e0c44e1a0bd3beaff37e4b4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:44:22.464654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:22.478650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebbdb6-661a-4f70-b67e-cbf6d35b3bf4 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:22.503534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48d519-f8a5-42c5-8bbe-0fe291bc2338 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:22.519079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e473298-04b7-49d9-9ac4-d436a9d30b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:22.519725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e473298-04b7-49d9-9ac4-d436a9d30b16 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:22.520015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e473298-04b7-49d9-9ac4-d436a9d30b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:22.520290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e473298-04b7-49d9-9ac4-d436a9d30b16 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:22.521029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e473298-04b7-49d9-9ac4-d436a9d30b16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:22.521337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e473298-04b7-49d9-9ac4-d436a9d30b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:22.522738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e473298-04b7-49d9-9ac4-d436a9d30b16 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:22.533228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 538/1078] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:44:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:22.540883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e473298-04b7-49d9-9ac4-d436a9d30b16 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:22.629791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48d519-f8a5-42c5-8bbe-0fe291bc2338 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:22.652353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:22.657973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e473298-04b7-49d9-9ac4-d436a9d30b16 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:22.671981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2b98c-219b-409f-baf3-7d50494b14ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:22.672738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2b98c-219b-409f-baf3-7d50494b14ee None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:22.673051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2b98c-219b-409f-baf3-7d50494b14ee None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:22.673709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2b98c-219b-409f-baf3-7d50494b14ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:22.696976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5424c-c464-4abe-b64d-40c4850e3dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:22.697614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5424c-c464-4abe-b64d-40c4850e3dca None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:22.698478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5424c-c464-4abe-b64d-40c4850e3dca None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:22.698881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5424c-c464-4abe-b64d-40c4850e3dca None admin] PATH_INFO: `/v3/users/18bf9d7dfbf34d64a11b5d6101c28a8d` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:22.700516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5424c-c464-4abe-b64d-40c4850e3dca None admin] RBAC: Authorizing `identity:get_user(user_id=18bf9d7dfbf34d64a11b5d6101c28a8d)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:22.701982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5424c-c464-4abe-b64d-40c4850e3dca None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:22.703718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 539/1079] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:22 2022] GET /identity/v3/users/18bf9d7dfbf34d64a11b5d6101c28a8d => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:22.709116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5424c-c464-4abe-b64d-40c4850e3dca None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:22.729833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:22.748455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2b98c-219b-409f-baf3-7d50494b14ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:22.749944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2b98c-219b-409f-baf3-7d50494b14ee None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:22.823785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5424c-c464-4abe-b64d-40c4850e3dca None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:22.862776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 541/1080] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 10:44:22.863918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2b98c-219b-409f-baf3-7d50494b14ee None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:22.866054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098abf-65f1-4e36-9974-204ff5de2746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:22.868115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098abf-65f1-4e36-9974-204ff5de2746 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:22.868115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098abf-65f1-4e36-9974-204ff5de2746 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:22.868115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098abf-65f1-4e36-9974-204ff5de2746 None admin] PATH_INFO: `/v3/projects/b240c81156184288a3f1f540836f195b` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:22.870080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e098abf-65f1-4e36-9974-204ff5de2746 None admin] RBAC: Authorizing `identity:get_project(project_id=b240c81156184288a3f1f540836f195b)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:22.871553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e098abf-65f1-4e36-9974-204ff5de2746 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:22.874314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 540/1081] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:44:22 2022] GET /identity/v3/projects/b240c81156184288a3f1f540836f195b => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:22.879899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098abf-65f1-4e36-9974-204ff5de2746 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:22.886817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:22.974137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:22.987384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2b98c-219b-409f-baf3-7d50494b14ee None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:22.992064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098abf-65f1-4e36-9974-204ff5de2746 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:23.021961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946f2bf-38c8-4580-822a-467da1eec85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:23.022589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946f2bf-38c8-4580-822a-467da1eec85e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:23.022870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946f2bf-38c8-4580-822a-467da1eec85e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:23.023139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946f2bf-38c8-4580-822a-467da1eec85e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:23.029450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f80fc-f12e-4f96-8dcf-9d6f9211c296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:23.030115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f80fc-f12e-4f96-8dcf-9d6f9211c296 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:23.030397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f80fc-f12e-4f96-8dcf-9d6f9211c296 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:23.030668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f80fc-f12e-4f96-8dcf-9d6f9211c296 None admin] PATH_INFO: `/v3/projects/b240c81156184288a3f1f540836f195b/users/18bf9d7dfbf34d64a11b5d6101c28a8d/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:23.032619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f80fc-f12e-4f96-8dcf-9d6f9211c296 None admin] RBAC: Authorizing `identity:create_grant(project_id=b240c81156184288a3f1f540836f195b, user_id=18bf9d7dfbf34d64a11b5d6101c28a8d, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:23.033789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946f2bf-38c8-4580-822a-467da1eec85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:23.034114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f80fc-f12e-4f96-8dcf-9d6f9211c296 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:23.035239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946f2bf-38c8-4580-822a-467da1eec85e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:23.056137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 541/1082] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:44:22 2022] PUT /identity/v3/projects/b240c81156184288a3f1f540836f195b/users/18bf9d7dfbf34d64a11b5d6101c28a8d/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0946f2bf-38c8-4580-822a-467da1eec85e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:23.056383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:23.057949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:23.058993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:23.058993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:23.058993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:23.058993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 542/1083] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:23.060650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f80fc-f12e-4f96-8dcf-9d6f9211c296 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:23.178595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f80fc-f12e-4f96-8dcf-9d6f9211c296 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:23.221516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e9ff-be12-45ac-8f1d-1e83d6586c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:23.221516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e9ff-be12-45ac-8f1d-1e83d6586c7f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:23.221516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e9ff-be12-45ac-8f1d-1e83d6586c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:23.221516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e9ff-be12-45ac-8f1d-1e83d6586c7f None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:23.222047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e9ff-be12-45ac-8f1d-1e83d6586c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:23.222047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e9ff-be12-45ac-8f1d-1e83d6586c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:23.223401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e9ff-be12-45ac-8f1d-1e83d6586c7f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:23.236555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 542/1084] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:24.575089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c89e1-7981-46fa-91e3-ea4ac010c444 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:24.575604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c89e1-7981-46fa-91e3-ea4ac010c444 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:24.575604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c89e1-7981-46fa-91e3-ea4ac010c444 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:24.576989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 543/1085] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:44:24.582503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c499b1f-3296-4081-ae67-55430539c474 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:24.582942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c499b1f-3296-4081-ae67-55430539c474 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:24.583043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c499b1f-3296-4081-ae67-55430539c474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:24.596832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:24.602681 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-0c499b1f-3296-4081-ae67-55430539c474 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:24.604219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0c499b1f-3296-4081-ae67-55430539c474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:24.671150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 543/1086] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:24.944641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c89e1-7981-46fa-91e3-ea4ac010c444 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.015189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:25.048406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c89e1-7981-46fa-91e3-ea4ac010c444 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:25.083547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36005709-efcf-437f-a7fe-0f15765cd63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:25.084218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36005709-efcf-437f-a7fe-0f15765cd63f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.084495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36005709-efcf-437f-a7fe-0f15765cd63f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.084765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36005709-efcf-437f-a7fe-0f15765cd63f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.097636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36005709-efcf-437f-a7fe-0f15765cd63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-36005709-efcf-437f-a7fe-0f15765cd63f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:25.102716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:25.104060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:25.105189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 544/1087] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Sep 15 10:44:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:25.110522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c499b1f-3296-4081-ae67-55430539c474 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.201748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dd17e-c066-4e6a-99f2-99f3a39fd20e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.201945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dd17e-c066-4e6a-99f2-99f3a39fd20e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.202216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dd17e-c066-4e6a-99f2-99f3a39fd20e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.203813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 545/1088] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:44:25.208684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e36ece-92ad-4b82-ac60-d7fb743a4821 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.209003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e36ece-92ad-4b82-ac60-d7fb743a4821 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.209285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e36ece-92ad-4b82-ac60-d7fb743a4821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.210420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c499b1f-3296-4081-ae67-55430539c474 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:25.227766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-51e36ece-92ad-4b82-ac60-d7fb743a4821 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:25.229826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-51e36ece-92ad-4b82-ac60-d7fb743a4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:25.246374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b42fa5-78e9-428d-86c5-4101728ed7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:25.246986 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42fa5-78e9-428d-86c5-4101728ed7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.247275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42fa5-78e9-428d-86c5-4101728ed7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.247553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42fa5-78e9-428d-86c5-4101728ed7c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.248214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b42fa5-78e9-428d-86c5-4101728ed7c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:25.248521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b42fa5-78e9-428d-86c5-4101728ed7c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:25.249932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b42fa5-78e9-428d-86c5-4101728ed7c8 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:25.252154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:25.258567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 544/1089] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Sep 15 10:44:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:25.264505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b42fa5-78e9-428d-86c5-4101728ed7c8 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.298269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 546/1090] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:44:25.362810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b42fa5-78e9-428d-86c5-4101728ed7c8 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:25.395400 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d941c-1621-45ed-a8a4-ba35d134cb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:25.396056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d941c-1621-45ed-a8a4-ba35d134cb4b None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.396335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d941c-1621-45ed-a8a4-ba35d134cb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.396605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d941c-1621-45ed-a8a4-ba35d134cb4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.407756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d941c-1621-45ed-a8a4-ba35d134cb4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:25.409368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d941c-1621-45ed-a8a4-ba35d134cb4b None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-a02d941c-1621-45ed-a8a4-ba35d134cb4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:25.424192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:25.425416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:25.426372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:25.426372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:25.426372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:25.426372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 545/1091] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:25.431212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e36ece-92ad-4b82-ac60-d7fb743a4821 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.488838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:25.528549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e36ece-92ad-4b82-ac60-d7fb743a4821 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:25.568307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac6b6f-98c4-4800-86a9-f2ce7e650847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:25.568955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac6b6f-98c4-4800-86a9-f2ce7e650847 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.569237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac6b6f-98c4-4800-86a9-f2ce7e650847 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.569508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac6b6f-98c4-4800-86a9-f2ce7e650847 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.570147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac6b6f-98c4-4800-86a9-f2ce7e650847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:25.570450 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac6b6f-98c4-4800-86a9-f2ce7e650847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:25.571834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac6b6f-98c4-4800-86a9-f2ce7e650847 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:25.573599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d941c-1621-45ed-a8a4-ba35d134cb4b None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.584717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 547/1092] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:25.590292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac6b6f-98c4-4800-86a9-f2ce7e650847 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.628649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:25.677977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d941c-1621-45ed-a8a4-ba35d134cb4b None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:25.689566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac6b6f-98c4-4800-86a9-f2ce7e650847 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:25.709716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:25.712361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961c8a9-f05e-4215-a890-6f29ed171bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:25.713033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961c8a9-f05e-4215-a890-6f29ed171bda None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.713334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961c8a9-f05e-4215-a890-6f29ed171bda None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.713789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961c8a9-f05e-4215-a890-6f29ed171bda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.724216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1d5fb-f564-49a4-9ca5-2a9691fd84b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:25.724389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961c8a9-f05e-4215-a890-6f29ed171bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:25.724860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1d5fb-f564-49a4-9ca5-2a9691fd84b2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.725143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1d5fb-f564-49a4-9ca5-2a9691fd84b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.725414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1d5fb-f564-49a4-9ca5-2a9691fd84b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-3961c8a9-f05e-4215-a890-6f29ed171bda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:25.729123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:25.730220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:25.731349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 546/1093] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:44:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:25.733099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1d5fb-f564-49a4-9ca5-2a9691fd84b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:25.734548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1d5fb-f564-49a4-9ca5-2a9691fd84b2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:25.736311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961c8a9-f05e-4215-a890-6f29ed171bda None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-dac1d5fb-f564-49a4-9ca5-2a9691fd84b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:44:25.746368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:25.747578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:25.749114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 548/1094] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Sep 15 10:44:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:25.753169 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1d5fb-f564-49a4-9ca5-2a9691fd84b2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.840862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961c8a9-f05e-4215-a890-6f29ed171bda None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:25.850977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:25.854627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1d5fb-f564-49a4-9ca5-2a9691fd84b2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:25.875912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935ea5f-a457-4c53-8863-96858c49f6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:25.876513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935ea5f-a457-4c53-8863-96858c49f6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.876795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935ea5f-a457-4c53-8863-96858c49f6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.877101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935ea5f-a457-4c53-8863-96858c49f6a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.877761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935ea5f-a457-4c53-8863-96858c49f6a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:25.878068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935ea5f-a457-4c53-8863-96858c49f6a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:25.879447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935ea5f-a457-4c53-8863-96858c49f6a6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:25.888995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 547/1095] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:44:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:25.889211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa4e0-1f4c-4d19-b2fe-dfb42d23cd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:25.889820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa4e0-1f4c-4d19-b2fe-dfb42d23cd52 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:25.890102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa4e0-1f4c-4d19-b2fe-dfb42d23cd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:25.890378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa4e0-1f4c-4d19-b2fe-dfb42d23cd52 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:25.891101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fa4e0-1f4c-4d19-b2fe-dfb42d23cd52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:25.891406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fa4e0-1f4c-4d19-b2fe-dfb42d23cd52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:25.892849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fa4e0-1f4c-4d19-b2fe-dfb42d23cd52 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:25.894884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935ea5f-a457-4c53-8863-96858c49f6a6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.910812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 549/1096] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Sep 15 10:44:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:25.916426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa4e0-1f4c-4d19-b2fe-dfb42d23cd52 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:25.955940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:25.996753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935ea5f-a457-4c53-8863-96858c49f6a6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:26.015259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa4e0-1f4c-4d19-b2fe-dfb42d23cd52 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:26.030909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3af99e-b4f6-4343-9195-d31d82eb88e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:26.031536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3af99e-b4f6-4343-9195-d31d82eb88e3 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:26.031814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3af99e-b4f6-4343-9195-d31d82eb88e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:26.032089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3af99e-b4f6-4343-9195-d31d82eb88e3 None admin] PATH_INFO: `/v3/users/18bf9d7dfbf34d64a11b5d6101c28a8d` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:26.033244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3af99e-b4f6-4343-9195-d31d82eb88e3 None admin] RBAC: Authorizing `identity:get_user(user_id=18bf9d7dfbf34d64a11b5d6101c28a8d)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:26.034680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3af99e-b4f6-4343-9195-d31d82eb88e3 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:26.036257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 548/1097] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:25 2022] GET /identity/v3/users/18bf9d7dfbf34d64a11b5d6101c28a8d => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:26.042040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3af99e-b4f6-4343-9195-d31d82eb88e3 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:26.049604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57fe22-ac2a-419b-a2a0-3d742fcc1607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:26.050226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57fe22-ac2a-419b-a2a0-3d742fcc1607 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:26.050510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57fe22-ac2a-419b-a2a0-3d742fcc1607 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:26.050784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57fe22-ac2a-419b-a2a0-3d742fcc1607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:26.084630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:26.118817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57fe22-ac2a-419b-a2a0-3d742fcc1607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:26.120850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57fe22-ac2a-419b-a2a0-3d742fcc1607 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:26.143959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3af99e-b4f6-4343-9195-d31d82eb88e3 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:26.158469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:26.178350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dcb98-1717-4ab8-b908-b00d48f843f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:26.178950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dcb98-1717-4ab8-b908-b00d48f843f6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:26.179228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dcb98-1717-4ab8-b908-b00d48f843f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:26.179499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dcb98-1717-4ab8-b908-b00d48f843f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:26.220417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 550/1098] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 10:44:26.226212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57fe22-ac2a-419b-a2a0-3d742fcc1607 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:26.248836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dcb98-1717-4ab8-b908-b00d48f843f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:26.250224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dcb98-1717-4ab8-b908-b00d48f843f6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:26.289033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:26.330560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57fe22-ac2a-419b-a2a0-3d742fcc1607 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:26.348345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 549/1099] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 10:44:26.354951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dcb98-1717-4ab8-b908-b00d48f843f6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:26.367710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c312956-ceea-4237-8a69-30271a613a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:26.368350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312956-ceea-4237-8a69-30271a613a27 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:26.368640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312956-ceea-4237-8a69-30271a613a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:26.368948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312956-ceea-4237-8a69-30271a613a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:26.376134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:26.380691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c312956-ceea-4237-8a69-30271a613a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:26.382188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c312956-ceea-4237-8a69-30271a613a27 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6c312956-ceea-4237-8a69-30271a613a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:26.396470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:26.397629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:44:26.398575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:26.398575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:44:26.398575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:26.398575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 551/1100] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:26.403900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c312956-ceea-4237-8a69-30271a613a27 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:26.458376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dcb98-1717-4ab8-b908-b00d48f843f6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:26.492400 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45c8f7-ea04-4e84-8e38-a4efb3925fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:26.493047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45c8f7-ea04-4e84-8e38-a4efb3925fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:26.493328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45c8f7-ea04-4e84-8e38-a4efb3925fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:26.493597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45c8f7-ea04-4e84-8e38-a4efb3925fe9 None admin] PATH_INFO: `/v3/projects/b240c81156184288a3f1f540836f195b` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:26.495332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45c8f7-ea04-4e84-8e38-a4efb3925fe9 None admin] RBAC: Authorizing `identity:get_project(project_id=b240c81156184288a3f1f540836f195b)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:26.497047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45c8f7-ea04-4e84-8e38-a4efb3925fe9 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:26.498694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 550/1101] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:44:26 2022] GET /identity/v3/projects/b240c81156184288a3f1f540836f195b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:26.502970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c312956-ceea-4237-8a69-30271a613a27 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:26.504688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45c8f7-ea04-4e84-8e38-a4efb3925fe9 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:26.517706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:26.536913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aa02f-7a40-4915-8d26-2865c97fec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:26.537551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aa02f-7a40-4915-8d26-2865c97fec3b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:26.537837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aa02f-7a40-4915-8d26-2865c97fec3b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:26.538111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aa02f-7a40-4915-8d26-2865c97fec3b None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:26.538745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2aa02f-7a40-4915-8d26-2865c97fec3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:26.539049 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2aa02f-7a40-4915-8d26-2865c97fec3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:26.540426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2aa02f-7a40-4915-8d26-2865c97fec3b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:26.553058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 552/1102] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:26.559565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aa02f-7a40-4915-8d26-2865c97fec3b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:26.609335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:26.612272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45c8f7-ea04-4e84-8e38-a4efb3925fe9 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:26.647690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07998e6-c6b1-4f09-8f53-881d0e35f423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:26.648322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07998e6-c6b1-4f09-8f53-881d0e35f423 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:26.648553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07998e6-c6b1-4f09-8f53-881d0e35f423 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:26.648851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07998e6-c6b1-4f09-8f53-881d0e35f423 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:26.649565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07998e6-c6b1-4f09-8f53-881d0e35f423 None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=b240c81156184288a3f1f540836f195b, user.id=18bf9d7dfbf34d64a11b5d6101c28a8d) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:26.649871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07998e6-c6b1-4f09-8f53-881d0e35f423 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:26.652182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07998e6-c6b1-4f09-8f53-881d0e35f423 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:26.657831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 551/1103] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:44:26 2022] GET /identity/v3/role_assignments?user.id=18bf9d7dfbf34d64a11b5d6101c28a8d&scope.project.id=b240c81156184288a3f1f540836f195b&role.id=0be9777002c641b4a47f78df647cc173 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:26.660821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aa02f-7a40-4915-8d26-2865c97fec3b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:26.700044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeb0d2-8c87-4a2f-8483-de3976922efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:26.700700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeb0d2-8c87-4a2f-8483-de3976922efd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:26.701013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeb0d2-8c87-4a2f-8483-de3976922efd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:26.701286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeb0d2-8c87-4a2f-8483-de3976922efd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:26.708728 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeb0d2-8c87-4a2f-8483-de3976922efd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:26.710302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeb0d2-8c87-4a2f-8483-de3976922efd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0ebeb0d2-8c87-4a2f-8483-de3976922efd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:44:26.722796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:26.723879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:26.725248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:44:26.725248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:44:26.725248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:44:26.725248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:44:26.725248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:44:26.725248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:44:26.725248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:44:26.725248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:26.725248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 553/1104] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:44:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:44:26.730946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07998e6-c6b1-4f09-8f53-881d0e35f423 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:26.805674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:26.840282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07998e6-c6b1-4f09-8f53-881d0e35f423 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:26.873892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e128a-9c08-42e7-b939-e9f24fdf5b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:26.874495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e128a-9c08-42e7-b939-e9f24fdf5b34 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:26.874773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e128a-9c08-42e7-b939-e9f24fdf5b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:26.875038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e128a-9c08-42e7-b939-e9f24fdf5b34 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:26.875721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e128a-9c08-42e7-b939-e9f24fdf5b34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:26.876020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e128a-9c08-42e7-b939-e9f24fdf5b34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:26.877469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e128a-9c08-42e7-b939-e9f24fdf5b34 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:26.890947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 552/1105] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:44:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:26.897274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeb0d2-8c87-4a2f-8483-de3976922efd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:26.996686 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeb0d2-8c87-4a2f-8483-de3976922efd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:27.007157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:27.032073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb160ceb-5f85-42f1-99a9-df91e1555ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:27.032677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb160ceb-5f85-42f1-99a9-df91e1555ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:27.032978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb160ceb-5f85-42f1-99a9-df91e1555ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:27.033250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb160ceb-5f85-42f1-99a9-df91e1555ea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:27.033962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb160ceb-5f85-42f1-99a9-df91e1555ea6 None admin] RBAC: Adding query filter params (role.id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=218a015dd2b34435a76ecc8a763562ed) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:27.034265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb160ceb-5f85-42f1-99a9-df91e1555ea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:27.035689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb160ceb-5f85-42f1-99a9-df91e1555ea6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:27.041765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 554/1106] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:44:26 2022] GET /identity/v3/role_assignments?user.id=218a015dd2b34435a76ecc8a763562ed&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=b4c8a5ff4e0c44e1a0bd3beaff37e4b4 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:28.839839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876981df-9dc4-4f9d-b659-b278b7d9e072 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:28.840342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876981df-9dc4-4f9d-b659-b278b7d9e072 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:28.840342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876981df-9dc4-4f9d-b659-b278b7d9e072 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:28.841685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 553/1107] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:28.846806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52df03-5557-4f2a-824a-5977f4f6ce0a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:28.847188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52df03-5557-4f2a-824a-5977f4f6ce0a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:28.847365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52df03-5557-4f2a-824a-5977f4f6ce0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:28.865782 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-eb52df03-5557-4f2a-824a-5977f4f6ce0a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:28.866968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eb52df03-5557-4f2a-824a-5977f4f6ce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:28.931659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 555/1108] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:29.203283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876981df-9dc4-4f9d-b659-b278b7d9e072 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:29.304539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876981df-9dc4-4f9d-b659-b278b7d9e072 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:29.333633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:29.339522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ddeba-3898-4052-bfa1-b13586a9e2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:29.340148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ddeba-3898-4052-bfa1-b13586a9e2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:29.340426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ddeba-3898-4052-bfa1-b13586a9e2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:29.340692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ddeba-3898-4052-bfa1-b13586a9e2a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:29.341857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ddeba-3898-4052-bfa1-b13586a9e2a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:29.343284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ddeba-3898-4052-bfa1-b13586a9e2a1 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:29.344858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 554/1109] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:29.350551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52df03-5557-4f2a-824a-5977f4f6ce0a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:29.446721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52df03-5557-4f2a-824a-5977f4f6ce0a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:29.449963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:29.482910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fb573-4745-4581-aad1-7575a22ee8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:29.483447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fb573-4745-4581-aad1-7575a22ee8ff None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:29.483727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fb573-4745-4581-aad1-7575a22ee8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:29.483995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fb573-4745-4581-aad1-7575a22ee8ff None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:29.484641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4fb573-4745-4581-aad1-7575a22ee8ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:29.486100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4fb573-4745-4581-aad1-7575a22ee8ff None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:29.506966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 556/1110] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:44:29 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:31.842011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb73f9-524b-4acd-b5cc-64e9dfb7bb4b None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:31.842536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb73f9-524b-4acd-b5cc-64e9dfb7bb4b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:31.842536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb73f9-524b-4acd-b5cc-64e9dfb7bb4b None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:31.843798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 555/1111] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:31.849074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a086575-2ee9-45ac-85aa-b68f811daf1f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:31.849468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a086575-2ee9-45ac-85aa-b68f811daf1f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:31.849626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a086575-2ee9-45ac-85aa-b68f811daf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:31.867260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3a086575-2ee9-45ac-85aa-b68f811daf1f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:31.868428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3a086575-2ee9-45ac-85aa-b68f811daf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:31.935047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 557/1112] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:32.215357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb73f9-524b-4acd-b5cc-64e9dfb7bb4b None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:32.316764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb73f9-524b-4acd-b5cc-64e9dfb7bb4b None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:32.350234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f18e1a-7500-4ee7-8bb7-95ac9dd9addb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:32.350852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f18e1a-7500-4ee7-8bb7-95ac9dd9addb None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:32.351130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f18e1a-7500-4ee7-8bb7-95ac9dd9addb None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:32.351400 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f18e1a-7500-4ee7-8bb7-95ac9dd9addb None admin] PATH_INFO: `/v3/roles/3518baf98ba54a28a9c79ec287469aff` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:32.352773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f18e1a-7500-4ee7-8bb7-95ac9dd9addb None admin] RBAC: Authorizing `identity:get_role(role_id=3518baf98ba54a28a9c79ec287469aff)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:32.354861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f18e1a-7500-4ee7-8bb7-95ac9dd9addb None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:32.356162 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 556/1113] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:32 2022] GET /identity/v3/roles/3518baf98ba54a28a9c79ec287469aff => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:32.361986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a086575-2ee9-45ac-85aa-b68f811daf1f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:32.451708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:32.462005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a086575-2ee9-45ac-85aa-b68f811daf1f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:32.495612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599389c-9f74-45bf-a85e-974358203ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:32.496177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599389c-9f74-45bf-a85e-974358203ccb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:32.496458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599389c-9f74-45bf-a85e-974358203ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:32.496729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599389c-9f74-45bf-a85e-974358203ccb None admin] PATH_INFO: `/v3/users/24f7455ed3db4d368d1a8810d044e63f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:32.510851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599389c-9f74-45bf-a85e-974358203ccb None admin] RBAC: Authorizing `identity:get_user(user_id=24f7455ed3db4d368d1a8810d044e63f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:32.512297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599389c-9f74-45bf-a85e-974358203ccb None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:32.513925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 558/1114] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:32 2022] GET /identity/v3/users/24f7455ed3db4d368d1a8810d044e63f => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:32.519686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f18e1a-7500-4ee7-8bb7-95ac9dd9addb None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:32.585264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:32.617363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f18e1a-7500-4ee7-8bb7-95ac9dd9addb None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:32.650195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f0169-125f-4a3b-bf08-59ea1545e8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:32.650854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f0169-125f-4a3b-bf08-59ea1545e8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:32.651134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f0169-125f-4a3b-bf08-59ea1545e8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:32.651406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f0169-125f-4a3b-bf08-59ea1545e8b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:32.716826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f0169-125f-4a3b-bf08-59ea1545e8b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:32.718264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f0169-125f-4a3b-bf08-59ea1545e8b7 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:32.779696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:32.812635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 557/1115] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:44:32.818616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599389c-9f74-45bf-a85e-974358203ccb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:32.915562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599389c-9f74-45bf-a85e-974358203ccb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:32.949099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b1ef0-9f31-49e3-8e39-1c2627172949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:32.949972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b1ef0-9f31-49e3-8e39-1c2627172949 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:32.950250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b1ef0-9f31-49e3-8e39-1c2627172949 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:32.950519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b1ef0-9f31-49e3-8e39-1c2627172949 None admin] PATH_INFO: `/v3/projects/b240c81156184288a3f1f540836f195b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:32.951611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b1ef0-9f31-49e3-8e39-1c2627172949 None admin] RBAC: Authorizing `identity:get_project(project_id=b240c81156184288a3f1f540836f195b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:32.953068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b1ef0-9f31-49e3-8e39-1c2627172949 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:32.954682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 559/1116] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:44:32 2022] GET /identity/v3/projects/b240c81156184288a3f1f540836f195b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:32.960651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f0169-125f-4a3b-bf08-59ea1545e8b7 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:33.057739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f0169-125f-4a3b-bf08-59ea1545e8b7 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:33.091033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64159d2-2b20-433d-ab6d-bb4c0dc88591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:33.091617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64159d2-2b20-433d-ab6d-bb4c0dc88591 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:33.091891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64159d2-2b20-433d-ab6d-bb4c0dc88591 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:33.092157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64159d2-2b20-433d-ab6d-bb4c0dc88591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:33.092891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64159d2-2b20-433d-ab6d-bb4c0dc88591 None admin] RBAC: Adding query filter params (role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=b240c81156184288a3f1f540836f195b, user.id=24f7455ed3db4d368d1a8810d044e63f) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:33.093196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64159d2-2b20-433d-ab6d-bb4c0dc88591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:33.094604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64159d2-2b20-433d-ab6d-bb4c0dc88591 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:33.099749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 558/1117] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:44:32 2022] GET /identity/v3/role_assignments?user.id=24f7455ed3db4d368d1a8810d044e63f&scope.project.id=b240c81156184288a3f1f540836f195b&role.id=3518baf98ba54a28a9c79ec287469aff => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:35.216710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064edd2-24b0-4a41-b07e-04e8ab7fa6b0 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:35.217304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064edd2-24b0-4a41-b07e-04e8ab7fa6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:35.217304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064edd2-24b0-4a41-b07e-04e8ab7fa6b0 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:35.218638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 560/1118] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:35.223692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5146f-9ccd-4828-acc9-7d04dd0c6210 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:35.224115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5146f-9ccd-4828-acc9-7d04dd0c6210 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:35.224242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5146f-9ccd-4828-acc9-7d04dd0c6210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:35.242967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-32c5146f-9ccd-4828-acc9-7d04dd0c6210 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:35.244630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-32c5146f-9ccd-4828-acc9-7d04dd0c6210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:35.270259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:35.311736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 559/1119] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:35.581768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064edd2-24b0-4a41-b07e-04e8ab7fa6b0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:35.599434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:35.682602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064edd2-24b0-4a41-b07e-04e8ab7fa6b0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:35.714743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded4836-0cac-4f31-b8e6-88d0c0a96d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:35.715361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4836-0cac-4f31-b8e6-88d0c0a96d03 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:35.715637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4836-0cac-4f31-b8e6-88d0c0a96d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:35.715903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4836-0cac-4f31-b8e6-88d0c0a96d03 None admin] PATH_INFO: `/v3/roles/3518baf98ba54a28a9c79ec287469aff` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:35.717292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded4836-0cac-4f31-b8e6-88d0c0a96d03 None admin] RBAC: Authorizing `identity:get_role(role_id=3518baf98ba54a28a9c79ec287469aff)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:35.718649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded4836-0cac-4f31-b8e6-88d0c0a96d03 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:35.719949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 561/1120] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:35 2022] GET /identity/v3/roles/3518baf98ba54a28a9c79ec287469aff => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:35.725760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5146f-9ccd-4828-acc9-7d04dd0c6210 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:35.825690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5146f-9ccd-4828-acc9-7d04dd0c6210 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:35.858229 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84a372-2548-457f-9292-834cf693c717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:35.858846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84a372-2548-457f-9292-834cf693c717 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:35.859123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84a372-2548-457f-9292-834cf693c717 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:35.859390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84a372-2548-457f-9292-834cf693c717 None admin] PATH_INFO: `/v3/users/24f7455ed3db4d368d1a8810d044e63f` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:35.860490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84a372-2548-457f-9292-834cf693c717 None admin] RBAC: Authorizing `identity:get_user(user_id=24f7455ed3db4d368d1a8810d044e63f)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:35.861954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84a372-2548-457f-9292-834cf693c717 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:35.863502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 560/1121] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:35 2022] GET /identity/v3/users/24f7455ed3db4d368d1a8810d044e63f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:35.869785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded4836-0cac-4f31-b8e6-88d0c0a96d03 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:35.952018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:35.967578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded4836-0cac-4f31-b8e6-88d0c0a96d03 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:36.000344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bf85a-6d89-4dbb-965f-afd8d1a80539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:36.001001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bf85a-6d89-4dbb-965f-afd8d1a80539 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:36.001281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bf85a-6d89-4dbb-965f-afd8d1a80539 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:36.001548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bf85a-6d89-4dbb-965f-afd8d1a80539 None admin] PATH_INFO: `/v3/projects/b240c81156184288a3f1f540836f195b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:36.002645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bf85a-6d89-4dbb-965f-afd8d1a80539 None admin] RBAC: Authorizing `identity:get_project(project_id=b240c81156184288a3f1f540836f195b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:36.004074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bf85a-6d89-4dbb-965f-afd8d1a80539 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:36.005700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 562/1122] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:44:35 2022] GET /identity/v3/projects/b240c81156184288a3f1f540836f195b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:36.012269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84a372-2548-457f-9292-834cf693c717 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:36.041652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:36.111108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84a372-2548-457f-9292-834cf693c717 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:36.143899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60edad3-6298-4a1f-8400-1f3377ff0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:36.144542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60edad3-6298-4a1f-8400-1f3377ff0129 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:36.144853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60edad3-6298-4a1f-8400-1f3377ff0129 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:36.145135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60edad3-6298-4a1f-8400-1f3377ff0129 None admin] PATH_INFO: `/v3/projects/b240c81156184288a3f1f540836f195b/users/24f7455ed3db4d368d1a8810d044e63f/roles/3518baf98ba54a28a9c79ec287469aff` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:36.147256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60edad3-6298-4a1f-8400-1f3377ff0129 None admin] RBAC: Authorizing `identity:create_grant(project_id=b240c81156184288a3f1f540836f195b, user_id=24f7455ed3db4d368d1a8810d044e63f, role_id=3518baf98ba54a28a9c79ec287469aff)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:36.148686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60edad3-6298-4a1f-8400-1f3377ff0129 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:36.161389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 561/1123] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:44:36 2022] PUT /identity/v3/projects/b240c81156184288a3f1f540836f195b/users/24f7455ed3db4d368d1a8810d044e63f/roles/3518baf98ba54a28a9c79ec287469aff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:44:38.289900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524bcfd-47f2-4637-b85b-6d07f0160562 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:38.290437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524bcfd-47f2-4637-b85b-6d07f0160562 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:38.290437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524bcfd-47f2-4637-b85b-6d07f0160562 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:38.292127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 563/1124] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:38.297608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0078529-42b4-4ed6-96ff-9ed31902bdd5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:38.298010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0078529-42b4-4ed6-96ff-9ed31902bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:38.298145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0078529-42b4-4ed6-96ff-9ed31902bdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:38.317280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-a0078529-42b4-4ed6-96ff-9ed31902bdd5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:38.318487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a0078529-42b4-4ed6-96ff-9ed31902bdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:38.382845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 562/1125] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:38.655283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524bcfd-47f2-4637-b85b-6d07f0160562 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:38.753808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524bcfd-47f2-4637-b85b-6d07f0160562 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:38.787156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0281b63-71bd-4741-9d25-0da37fd9955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:38.787776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0281b63-71bd-4741-9d25-0da37fd9955b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:38.788057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0281b63-71bd-4741-9d25-0da37fd9955b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:38.788329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0281b63-71bd-4741-9d25-0da37fd9955b None admin] PATH_INFO: `/v3/roles/3518baf98ba54a28a9c79ec287469aff` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:38.789702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0281b63-71bd-4741-9d25-0da37fd9955b None admin] RBAC: Authorizing `identity:get_role(role_id=3518baf98ba54a28a9c79ec287469aff)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:38.791342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0281b63-71bd-4741-9d25-0da37fd9955b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:38.792633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 564/1126] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:38 2022] GET /identity/v3/roles/3518baf98ba54a28a9c79ec287469aff => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:38.798320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0078529-42b4-4ed6-96ff-9ed31902bdd5 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:38.818902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:38.895570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0078529-42b4-4ed6-96ff-9ed31902bdd5 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:38.927949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c3885-1197-4361-b426-fbbeded9efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:38.928572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c3885-1197-4361-b426-fbbeded9efb3 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:38.928876 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c3885-1197-4361-b426-fbbeded9efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:38.929148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c3885-1197-4361-b426-fbbeded9efb3 None admin] PATH_INFO: `/v3/users/24f7455ed3db4d368d1a8810d044e63f` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:38.930251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c3885-1197-4361-b426-fbbeded9efb3 None admin] RBAC: Authorizing `identity:get_user(user_id=24f7455ed3db4d368d1a8810d044e63f)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:38.932181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c3885-1197-4361-b426-fbbeded9efb3 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:38.933774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 563/1127] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:38 2022] GET /identity/v3/users/24f7455ed3db4d368d1a8810d044e63f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:38.939550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0281b63-71bd-4741-9d25-0da37fd9955b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:38.961638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:39.036763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0281b63-71bd-4741-9d25-0da37fd9955b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:39.070469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa5fb1-b514-4a34-88ac-b360306b6614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:39.071001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa5fb1-b514-4a34-88ac-b360306b6614 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:39.071277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa5fb1-b514-4a34-88ac-b360306b6614 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:39.071546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa5fb1-b514-4a34-88ac-b360306b6614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:39.137182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa5fb1-b514-4a34-88ac-b360306b6614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:39.138623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa5fb1-b514-4a34-88ac-b360306b6614 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:39.158672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:39.234034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 565/1128] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:44:39.239902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c3885-1197-4361-b426-fbbeded9efb3 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:39.333143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:39.345036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c3885-1197-4361-b426-fbbeded9efb3 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:39.378312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6441f-9332-4f48-8b28-7e03e5e32d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:39.378927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6441f-9332-4f48-8b28-7e03e5e32d25 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:39.379204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6441f-9332-4f48-8b28-7e03e5e32d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:39.379470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6441f-9332-4f48-8b28-7e03e5e32d25 None admin] PATH_INFO: `/v3/projects/b240c81156184288a3f1f540836f195b` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:39.380558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6441f-9332-4f48-8b28-7e03e5e32d25 None admin] RBAC: Authorizing `identity:get_project(project_id=b240c81156184288a3f1f540836f195b)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:39.382015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6441f-9332-4f48-8b28-7e03e5e32d25 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:39.383614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 564/1129] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:44:39 2022] GET /identity/v3/projects/b240c81156184288a3f1f540836f195b => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:39.389789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa5fb1-b514-4a34-88ac-b360306b6614 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:39.486184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa5fb1-b514-4a34-88ac-b360306b6614 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:39.506587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:39.520871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f1555-512e-437f-932e-93ea64216d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:39.521470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1555-512e-437f-932e-93ea64216d17 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:39.521749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1555-512e-437f-932e-93ea64216d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:39.522018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1555-512e-437f-932e-93ea64216d17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:39.522730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f1555-512e-437f-932e-93ea64216d17 None admin] RBAC: Adding query filter params (role.id=3518baf98ba54a28a9c79ec287469aff, scope.project.id=b240c81156184288a3f1f540836f195b, user.id=24f7455ed3db4d368d1a8810d044e63f) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:39.523033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f1555-512e-437f-932e-93ea64216d17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:39.524500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f1555-512e-437f-932e-93ea64216d17 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:39.530100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 566/1130] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:44:39 2022] GET /identity/v3/role_assignments?user.id=24f7455ed3db4d368d1a8810d044e63f&scope.project.id=b240c81156184288a3f1f540836f195b&role.id=3518baf98ba54a28a9c79ec287469aff => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:41.682719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26a8f7-0490-4bb3-8d8f-cea1a5298642 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:41.683244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26a8f7-0490-4bb3-8d8f-cea1a5298642 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:41.683244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26a8f7-0490-4bb3-8d8f-cea1a5298642 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:41.684628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 565/1131] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:41.689790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fb828-d9ac-42e6-bfa1-601f62eeb932 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:41.690231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fb828-d9ac-42e6-bfa1-601f62eeb932 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:41.690325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fb828-d9ac-42e6-bfa1-601f62eeb932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:41.709040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5f1fb828-d9ac-42e6-bfa1-601f62eeb932 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:41.710214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5f1fb828-d9ac-42e6-bfa1-601f62eeb932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:41.776163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 567/1132] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:44:42.045846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26a8f7-0490-4bb3-8d8f-cea1a5298642 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:42.145027 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26a8f7-0490-4bb3-8d8f-cea1a5298642 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:42.178710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82a881-19ec-4f93-a38e-6450d091f70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:42.179326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82a881-19ec-4f93-a38e-6450d091f70a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:42.179605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82a881-19ec-4f93-a38e-6450d091f70a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:42.179877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82a881-19ec-4f93-a38e-6450d091f70a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:42.181525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82a881-19ec-4f93-a38e-6450d091f70a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:42.182952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82a881-19ec-4f93-a38e-6450d091f70a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:42.184514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 566/1133] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:42.190271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fb828-d9ac-42e6-bfa1-601f62eeb932 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:42.286667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fb828-d9ac-42e6-bfa1-601f62eeb932 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:42.300724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:42.320994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd90d0-06e2-4de9-9618-65548be0ac03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:42.321623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd90d0-06e2-4de9-9618-65548be0ac03 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:42.321903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd90d0-06e2-4de9-9618-65548be0ac03 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:42.322251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd90d0-06e2-4de9-9618-65548be0ac03 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:42.322906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd90d0-06e2-4de9-9618-65548be0ac03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:42.324583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd90d0-06e2-4de9-9618-65548be0ac03 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:42.349731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 568/1134] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:44:42 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:44:44.514462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c9a11-95d0-4a69-84e1-1fa82f6161fb None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:44.514998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c9a11-95d0-4a69-84e1-1fa82f6161fb None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:44.514998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c9a11-95d0-4a69-84e1-1fa82f6161fb None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:44.516332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 567/1135] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:44.521472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de754e8d-c4d9-450b-823b-f65ddbb3aad1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:44.521865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de754e8d-c4d9-450b-823b-f65ddbb3aad1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:44.522032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de754e8d-c4d9-450b-823b-f65ddbb3aad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:44.540987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-de754e8d-c4d9-450b-823b-f65ddbb3aad1 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:44.542258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-de754e8d-c4d9-450b-823b-f65ddbb3aad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:44.608406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 569/1136] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:44.880277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c9a11-95d0-4a69-84e1-1fa82f6161fb None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:44.905734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:44.982323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c9a11-95d0-4a69-84e1-1fa82f6161fb None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:45.014539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76efcf-2315-4cfb-a862-20a96323ddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:45.015155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76efcf-2315-4cfb-a862-20a96323ddca None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:45.015436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76efcf-2315-4cfb-a862-20a96323ddca None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:45.015707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76efcf-2315-4cfb-a862-20a96323ddca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:45.016830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76efcf-2315-4cfb-a862-20a96323ddca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:45.018261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76efcf-2315-4cfb-a862-20a96323ddca None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:45.019817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 568/1137] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:44:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:44:45.026061 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de754e8d-c4d9-450b-823b-f65ddbb3aad1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:45.117500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:45.124225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de754e8d-c4d9-450b-823b-f65ddbb3aad1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:45.158014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c856e8-2d14-4741-8f17-9c65717b232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:45.158613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c856e8-2d14-4741-8f17-9c65717b232c None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:45.158893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c856e8-2d14-4741-8f17-9c65717b232c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:45.159164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c856e8-2d14-4741-8f17-9c65717b232c None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:45.159805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c856e8-2d14-4741-8f17-9c65717b232c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:45.161286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c856e8-2d14-4741-8f17-9c65717b232c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:45.182194 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 570/1138] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:44:45 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:44:47.357175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171320f-e733-4ce9-ac84-5e904b71fa48 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:47.357704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171320f-e733-4ce9-ac84-5e904b71fa48 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:47.357704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171320f-e733-4ce9-ac84-5e904b71fa48 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:47.359071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 569/1139] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:47.364189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05010-6f86-4fe8-8e52-a7cb42318b44 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:47.364591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05010-6f86-4fe8-8e52-a7cb42318b44 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:47.364739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05010-6f86-4fe8-8e52-a7cb42318b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:47.384310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f8b05010-6f86-4fe8-8e52-a7cb42318b44 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:47.385598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f8b05010-6f86-4fe8-8e52-a7cb42318b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:47.450991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 571/1140] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:47.729312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171320f-e733-4ce9-ac84-5e904b71fa48 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:47.820457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:47.831266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171320f-e733-4ce9-ac84-5e904b71fa48 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:47.864210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4b20d-69ec-471c-8987-c2ed0c5f2125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:47.864856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4b20d-69ec-471c-8987-c2ed0c5f2125 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:47.865141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4b20d-69ec-471c-8987-c2ed0c5f2125 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:47.865413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4b20d-69ec-471c-8987-c2ed0c5f2125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:47.875778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4b20d-69ec-471c-8987-c2ed0c5f2125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-fad4b20d-69ec-471c-8987-c2ed0c5f2125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:47.880724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:47.881874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:47.882726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 570/1141] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:44:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:47.888489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05010-6f86-4fe8-8e52-a7cb42318b44 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:47.966463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:47.984935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05010-6f86-4fe8-8e52-a7cb42318b44 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:48.018008 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509d35c-bafd-4eb9-8bf2-ab4f7d135093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:48.018603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509d35c-bafd-4eb9-8bf2-ab4f7d135093 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:48.018878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509d35c-bafd-4eb9-8bf2-ab4f7d135093 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:48.019148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509d35c-bafd-4eb9-8bf2-ab4f7d135093 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:48.019796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509d35c-bafd-4eb9-8bf2-ab4f7d135093 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:48.020092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509d35c-bafd-4eb9-8bf2-ab4f7d135093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:48.021477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509d35c-bafd-4eb9-8bf2-ab4f7d135093 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:48.029740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 572/1142] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:44:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:48.035841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4b20d-69ec-471c-8987-c2ed0c5f2125 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:48.134797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4b20d-69ec-471c-8987-c2ed0c5f2125 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:48.168637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaad3f-f6c8-4d5c-8b9d-2e57002a2a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:48.169275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaad3f-f6c8-4d5c-8b9d-2e57002a2a47 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:48.169554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaad3f-f6c8-4d5c-8b9d-2e57002a2a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:48.169825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaad3f-f6c8-4d5c-8b9d-2e57002a2a47 None admin] PATH_INFO: `/v3/users/ae438119663e46e9af9d129b8f5f80df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:48.182987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfaad3f-f6c8-4d5c-8b9d-2e57002a2a47 None admin] RBAC: Authorizing `identity:get_user(user_id=ae438119663e46e9af9d129b8f5f80df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:48.184418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfaad3f-f6c8-4d5c-8b9d-2e57002a2a47 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:48.186048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 571/1143] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:48 2022] GET /identity/v3/users/ae438119663e46e9af9d129b8f5f80df => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:48.191781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509d35c-bafd-4eb9-8bf2-ab4f7d135093 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:48.290425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509d35c-bafd-4eb9-8bf2-ab4f7d135093 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:48.322403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcb43b-9845-42f5-abe2-0d4ebeba27af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:48.322996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcb43b-9845-42f5-abe2-0d4ebeba27af None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:48.323275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcb43b-9845-42f5-abe2-0d4ebeba27af None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:48.323544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcb43b-9845-42f5-abe2-0d4ebeba27af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:48.337328 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:48.389641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcb43b-9845-42f5-abe2-0d4ebeba27af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:48.391093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcb43b-9845-42f5-abe2-0d4ebeba27af None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:48.486536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 573/1144] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:44:48.492432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaad3f-f6c8-4d5c-8b9d-2e57002a2a47 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:48.505110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:48.592325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaad3f-f6c8-4d5c-8b9d-2e57002a2a47 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:48.625593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd1540-7eeb-46c7-9713-7e229f33ff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:48.626218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd1540-7eeb-46c7-9713-7e229f33ff19 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:48.626498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd1540-7eeb-46c7-9713-7e229f33ff19 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:48.626770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd1540-7eeb-46c7-9713-7e229f33ff19 None admin] PATH_INFO: `/v3/projects/7d1d03d7129f4768813d02e444485bac` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:48.627872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd1540-7eeb-46c7-9713-7e229f33ff19 None admin] RBAC: Authorizing `identity:get_project(project_id=7d1d03d7129f4768813d02e444485bac)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:48.629344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd1540-7eeb-46c7-9713-7e229f33ff19 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:48.631015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 572/1145] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:44:48 2022] GET /identity/v3/projects/7d1d03d7129f4768813d02e444485bac => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:48.637764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcb43b-9845-42f5-abe2-0d4ebeba27af None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:48.682336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:48.735643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcb43b-9845-42f5-abe2-0d4ebeba27af None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:48.768948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ae89b-8a64-45b0-8e41-7051a947682b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:48.769538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ae89b-8a64-45b0-8e41-7051a947682b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:48.769815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ae89b-8a64-45b0-8e41-7051a947682b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:48.770083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ae89b-8a64-45b0-8e41-7051a947682b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:48.770788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ae89b-8a64-45b0-8e41-7051a947682b None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=7d1d03d7129f4768813d02e444485bac, user.id=ae438119663e46e9af9d129b8f5f80df) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:48.771091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ae89b-8a64-45b0-8e41-7051a947682b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:48.772501 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ae89b-8a64-45b0-8e41-7051a947682b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:48.777902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 574/1146] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:44:48 2022] GET /identity/v3/role_assignments?user.id=ae438119663e46e9af9d129b8f5f80df&scope.project.id=7d1d03d7129f4768813d02e444485bac&role.id=0be9777002c641b4a47f78df647cc173 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:50.924650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c37fa-e3b7-442d-99a4-9606a2f1092f None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:50.925258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c37fa-e3b7-442d-99a4-9606a2f1092f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:50.925258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c37fa-e3b7-442d-99a4-9606a2f1092f None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:50.926554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 573/1147] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:50.932466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852f1fc-853b-45ed-8ff3-3a04a69a53cb None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:50.932907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852f1fc-853b-45ed-8ff3-3a04a69a53cb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:50.933038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852f1fc-853b-45ed-8ff3-3a04a69a53cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:50.951687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5852f1fc-853b-45ed-8ff3-3a04a69a53cb None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:50.952889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5852f1fc-853b-45ed-8ff3-3a04a69a53cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:51.017493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 575/1148] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:51.287224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c37fa-e3b7-442d-99a4-9606a2f1092f None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:51.365100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:51.387247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c37fa-e3b7-442d-99a4-9606a2f1092f None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:51.420604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b01fc-3bb1-4fbb-a65a-d6b792f7f013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:51.421265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b01fc-3bb1-4fbb-a65a-d6b792f7f013 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:51.421588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b01fc-3bb1-4fbb-a65a-d6b792f7f013 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:51.421897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b01fc-3bb1-4fbb-a65a-d6b792f7f013 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:51.432328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b01fc-3bb1-4fbb-a65a-d6b792f7f013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-c01b01fc-3bb1-4fbb-a65a-d6b792f7f013 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:51.437183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:51.438375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:51.439663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 574/1149] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:44:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:51.444426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852f1fc-853b-45ed-8ff3-3a04a69a53cb None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:51.483112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:51.543481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852f1fc-853b-45ed-8ff3-3a04a69a53cb None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:51.576947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a9f2f-dd74-47d2-81b9-41f38b03a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:51.577550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a9f2f-dd74-47d2-81b9-41f38b03a275 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:51.577827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a9f2f-dd74-47d2-81b9-41f38b03a275 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:51.578100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a9f2f-dd74-47d2-81b9-41f38b03a275 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:51.578757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a9f2f-dd74-47d2-81b9-41f38b03a275 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:51.579058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a9f2f-dd74-47d2-81b9-41f38b03a275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:51.580434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a9f2f-dd74-47d2-81b9-41f38b03a275 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:51.588319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 576/1150] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:44:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:51.594089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b01fc-3bb1-4fbb-a65a-d6b792f7f013 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:51.692086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b01fc-3bb1-4fbb-a65a-d6b792f7f013 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:51.724915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07f984-db1e-49f1-bfc1-a8cc0c06cc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:51.725533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07f984-db1e-49f1-bfc1-a8cc0c06cc05 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:51.725812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07f984-db1e-49f1-bfc1-a8cc0c06cc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:51.726084 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07f984-db1e-49f1-bfc1-a8cc0c06cc05 None admin] PATH_INFO: `/v3/users/ae438119663e46e9af9d129b8f5f80df` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:51.727857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07f984-db1e-49f1-bfc1-a8cc0c06cc05 None admin] RBAC: Authorizing `identity:get_user(user_id=ae438119663e46e9af9d129b8f5f80df)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:51.729352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07f984-db1e-49f1-bfc1-a8cc0c06cc05 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:51.730923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 575/1151] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:51 2022] GET /identity/v3/users/ae438119663e46e9af9d129b8f5f80df => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:51.736684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a9f2f-dd74-47d2-81b9-41f38b03a275 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:51.835923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:51.839100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a9f2f-dd74-47d2-81b9-41f38b03a275 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:51.872055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc939ed-889c-4467-abe0-5923a93f76c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:51.872672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc939ed-889c-4467-abe0-5923a93f76c3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:51.872979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc939ed-889c-4467-abe0-5923a93f76c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:51.873256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc939ed-889c-4467-abe0-5923a93f76c3 None admin] PATH_INFO: `/v3/projects/7d1d03d7129f4768813d02e444485bac` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:51.874351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc939ed-889c-4467-abe0-5923a93f76c3 None admin] RBAC: Authorizing `identity:get_project(project_id=7d1d03d7129f4768813d02e444485bac)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:51.875786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc939ed-889c-4467-abe0-5923a93f76c3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:51.877425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 577/1152] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:44:51 2022] GET /identity/v3/projects/7d1d03d7129f4768813d02e444485bac => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:51.884059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07f984-db1e-49f1-bfc1-a8cc0c06cc05 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:51.893295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:51.985000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07f984-db1e-49f1-bfc1-a8cc0c06cc05 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:52.017989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431a58b-db35-4e88-a4b7-5cbb9736a403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:52.018637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431a58b-db35-4e88-a4b7-5cbb9736a403 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:52.018919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431a58b-db35-4e88-a4b7-5cbb9736a403 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:52.019194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431a58b-db35-4e88-a4b7-5cbb9736a403 None admin] PATH_INFO: `/v3/projects/7d1d03d7129f4768813d02e444485bac/users/ae438119663e46e9af9d129b8f5f80df/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:52.021102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431a58b-db35-4e88-a4b7-5cbb9736a403 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d1d03d7129f4768813d02e444485bac, user_id=ae438119663e46e9af9d129b8f5f80df, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:52.022533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431a58b-db35-4e88-a4b7-5cbb9736a403 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:52.036161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 576/1153] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:44:51 2022] PUT /identity/v3/projects/7d1d03d7129f4768813d02e444485bac/users/ae438119663e46e9af9d129b8f5f80df/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:44:54.158629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7a2f1-efd7-4a11-87b6-6ff270eee83b None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:54.159158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7a2f1-efd7-4a11-87b6-6ff270eee83b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:54.159158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7a2f1-efd7-4a11-87b6-6ff270eee83b None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:54.160515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 578/1154] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:54.166621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108dc3c-8137-4338-9a9e-72b0fe34ec94 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:54.167026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108dc3c-8137-4338-9a9e-72b0fe34ec94 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:54.167169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108dc3c-8137-4338-9a9e-72b0fe34ec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:54.185386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-c108dc3c-8137-4338-9a9e-72b0fe34ec94 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:54.186562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c108dc3c-8137-4338-9a9e-72b0fe34ec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:54.245839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:54.254070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 577/1155] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:54.524685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7a2f1-efd7-4a11-87b6-6ff270eee83b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:54.625136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7a2f1-efd7-4a11-87b6-6ff270eee83b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:54.657834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20363f80-b368-4b7d-b2e8-f7a5e9e4a454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:54.658450 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20363f80-b368-4b7d-b2e8-f7a5e9e4a454 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:54.658729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20363f80-b368-4b7d-b2e8-f7a5e9e4a454 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:54.658999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20363f80-b368-4b7d-b2e8-f7a5e9e4a454 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:54.668753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20363f80-b368-4b7d-b2e8-f7a5e9e4a454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-20363f80-b368-4b7d-b2e8-f7a5e9e4a454 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:44:54.673604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:54.674725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:44:54.675612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 579/1156] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:44:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:54.681393 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108dc3c-8137-4338-9a9e-72b0fe34ec94 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:54.776540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108dc3c-8137-4338-9a9e-72b0fe34ec94 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:54.809926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafcb73-90e3-429e-85d5-13f463e22e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:54.810531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafcb73-90e3-429e-85d5-13f463e22e9d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:54.810841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafcb73-90e3-429e-85d5-13f463e22e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:54.811143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafcb73-90e3-429e-85d5-13f463e22e9d None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:54.811824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafcb73-90e3-429e-85d5-13f463e22e9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:54.812159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafcb73-90e3-429e-85d5-13f463e22e9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:54.813571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafcb73-90e3-429e-85d5-13f463e22e9d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:54.822059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 578/1157] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:44:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:54.828054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20363f80-b368-4b7d-b2e8-f7a5e9e4a454 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:54.856015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:54.926024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20363f80-b368-4b7d-b2e8-f7a5e9e4a454 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:54.959586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54dccb-3836-40a1-85ae-45e131efa859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:54.960198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54dccb-3836-40a1-85ae-45e131efa859 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:54.960476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54dccb-3836-40a1-85ae-45e131efa859 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:54.960749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54dccb-3836-40a1-85ae-45e131efa859 None admin] PATH_INFO: `/v3/users/ae438119663e46e9af9d129b8f5f80df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:54.961922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54dccb-3836-40a1-85ae-45e131efa859 None admin] RBAC: Authorizing `identity:get_user(user_id=ae438119663e46e9af9d129b8f5f80df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:54.963357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54dccb-3836-40a1-85ae-45e131efa859 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:54.964950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 580/1158] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:44:54 2022] GET /identity/v3/users/ae438119663e46e9af9d129b8f5f80df => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:54.970641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafcb73-90e3-429e-85d5-13f463e22e9d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:55.028336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:55.070296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafcb73-90e3-429e-85d5-13f463e22e9d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:55.104246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f200b1-e210-4f1d-b557-cf4aaae244bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:55.104869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f200b1-e210-4f1d-b557-cf4aaae244bf None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:55.105184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f200b1-e210-4f1d-b557-cf4aaae244bf None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:55.105488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f200b1-e210-4f1d-b557-cf4aaae244bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:55.169227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f200b1-e210-4f1d-b557-cf4aaae244bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:55.170656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f200b1-e210-4f1d-b557-cf4aaae244bf None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:55.225404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:55.268002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 579/1159] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:44:55.274078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54dccb-3836-40a1-85ae-45e131efa859 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:55.365375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:55.372274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54dccb-3836-40a1-85ae-45e131efa859 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:55.406257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa777fac-e8c0-48ad-b22c-68be3f205424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:55.406878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa777fac-e8c0-48ad-b22c-68be3f205424 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:55.407156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa777fac-e8c0-48ad-b22c-68be3f205424 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:55.407426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa777fac-e8c0-48ad-b22c-68be3f205424 None admin] PATH_INFO: `/v3/projects/7d1d03d7129f4768813d02e444485bac` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:55.408542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa777fac-e8c0-48ad-b22c-68be3f205424 None admin] RBAC: Authorizing `identity:get_project(project_id=7d1d03d7129f4768813d02e444485bac)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:55.410016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa777fac-e8c0-48ad-b22c-68be3f205424 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:55.411654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 581/1160] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:44:55 2022] GET /identity/v3/projects/7d1d03d7129f4768813d02e444485bac => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:55.418417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f200b1-e210-4f1d-b557-cf4aaae244bf None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:55.513954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f200b1-e210-4f1d-b557-cf4aaae244bf None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:55.547266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40932f2-d5c8-4ec0-b68c-2b6b70b343bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:55.547864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40932f2-d5c8-4ec0-b68c-2b6b70b343bc None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:55.548138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40932f2-d5c8-4ec0-b68c-2b6b70b343bc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:55.548405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40932f2-d5c8-4ec0-b68c-2b6b70b343bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:55.549146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40932f2-d5c8-4ec0-b68c-2b6b70b343bc None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=7d1d03d7129f4768813d02e444485bac, user.id=ae438119663e46e9af9d129b8f5f80df) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:55.549448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40932f2-d5c8-4ec0-b68c-2b6b70b343bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:55.550853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40932f2-d5c8-4ec0-b68c-2b6b70b343bc None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:55.557379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 580/1161] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:44:55 2022] GET /identity/v3/role_assignments?user.id=ae438119663e46e9af9d129b8f5f80df&scope.project.id=7d1d03d7129f4768813d02e444485bac&role.id=0be9777002c641b4a47f78df647cc173 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:44:57.709090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77209486-c216-4b80-8de0-7e481fdcfc24 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:57.709641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77209486-c216-4b80-8de0-7e481fdcfc24 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:57.709641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77209486-c216-4b80-8de0-7e481fdcfc24 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:57.711531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 582/1162] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:44:57.716903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b47ce-7138-42ab-bdcf-bce42d0655e0 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:57.717319 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b47ce-7138-42ab-bdcf-bce42d0655e0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:57.717460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b47ce-7138-42ab-bdcf-bce42d0655e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:57.736094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5a8b47ce-7138-42ab-bdcf-bce42d0655e0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:44:57.737325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5a8b47ce-7138-42ab-bdcf-bce42d0655e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:44:57.753394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:57.802941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 581/1163] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:44:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:44:58.071246 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77209486-c216-4b80-8de0-7e481fdcfc24 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:58.170262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77209486-c216-4b80-8de0-7e481fdcfc24 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:58.203707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cfc45-26c4-45e9-9d4d-f4a4bfdad973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:58.204299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cfc45-26c4-45e9-9d4d-f4a4bfdad973 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:58.204581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cfc45-26c4-45e9-9d4d-f4a4bfdad973 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:58.204881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cfc45-26c4-45e9-9d4d-f4a4bfdad973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:58.234723 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:58.271213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07cfc45-26c4-45e9-9d4d-f4a4bfdad973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:58.272673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07cfc45-26c4-45e9-9d4d-f4a4bfdad973 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:58.368745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 583/1164] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:44:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:44:58.374717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b47ce-7138-42ab-bdcf-bce42d0655e0 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:58.471046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b47ce-7138-42ab-bdcf-bce42d0655e0 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:58.503815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04880b2f-5455-425c-80ef-5531989cae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:58.504424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04880b2f-5455-425c-80ef-5531989cae4d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:58.504698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04880b2f-5455-425c-80ef-5531989cae4d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:58.504996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04880b2f-5455-425c-80ef-5531989cae4d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:58.511864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04880b2f-5455-425c-80ef-5531989cae4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:58.513337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04880b2f-5455-425c-80ef-5531989cae4d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-04880b2f-5455-425c-80ef-5531989cae4d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:44:58.525181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:44:58.526524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:44:58.527663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 582/1165] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:44:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:44:58.532158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cfc45-26c4-45e9-9d4d-f4a4bfdad973 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:44:58.594265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:44:58.631603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cfc45-26c4-45e9-9d4d-f4a4bfdad973 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:44:58.664511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943731c1-edd9-40b6-b314-ffd86a81e581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:44:58.665139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943731c1-edd9-40b6-b314-ffd86a81e581 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:44:58.665423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943731c1-edd9-40b6-b314-ffd86a81e581 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:44:58.665697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943731c1-edd9-40b6-b314-ffd86a81e581 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:44:58.666330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943731c1-edd9-40b6-b314-ffd86a81e581 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:44:58.666633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943731c1-edd9-40b6-b314-ffd86a81e581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:44:58.668006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943731c1-edd9-40b6-b314-ffd86a81e581 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:44:58.674991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 584/1166] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Sep 15 10:44:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:00.810204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e3f5-fe3e-4828-b227-cf8979594959 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:00.810716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e3f5-fe3e-4828-b227-cf8979594959 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:00.810716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e3f5-fe3e-4828-b227-cf8979594959 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:00.812011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 583/1167] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:45:00.817323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36251bf-b525-44f6-881d-75961d32db58 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:00.817691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36251bf-b525-44f6-881d-75961d32db58 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:00.817878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36251bf-b525-44f6-881d-75961d32db58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:00.836533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e36251bf-b525-44f6-881d-75961d32db58 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:45:00.837771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e36251bf-b525-44f6-881d-75961d32db58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:45:00.902750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 585/1168] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:45:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:45:01.171029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8e3f5-fe3e-4828-b227-cf8979594959 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:01.186277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:01.271636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8e3f5-fe3e-4828-b227-cf8979594959 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:01.304057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f37428-d532-4bfc-a5d5-631ff5434796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:01.304785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f37428-d532-4bfc-a5d5-631ff5434796 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:01.305099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f37428-d532-4bfc-a5d5-631ff5434796 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:01.305860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f37428-d532-4bfc-a5d5-631ff5434796 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:01.306533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f37428-d532-4bfc-a5d5-631ff5434796 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:01.307889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f37428-d532-4bfc-a5d5-631ff5434796 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:01.324092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 584/1169] 10.222.0.50 () {60 vars in 1138 bytes} [Thu Sep 15 10:45:01 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:45:03.461089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd9747-1e7e-442b-ada2-6d772a3ab158 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:03.461737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd9747-1e7e-442b-ada2-6d772a3ab158 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:03.461737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd9747-1e7e-442b-ada2-6d772a3ab158 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:03.463061 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 586/1170] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:45:03.468168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf687-bf07-4b63-8fa6-990878a98c09 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:03.468612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf687-bf07-4b63-8fa6-990878a98c09 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:03.468713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf687-bf07-4b63-8fa6-990878a98c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:03.486540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-77bdf687-bf07-4b63-8fa6-990878a98c09 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:45:03.487720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-77bdf687-bf07-4b63-8fa6-990878a98c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:45:03.553037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 585/1171] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:45:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:45:03.824322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd9747-1e7e-442b-ada2-6d772a3ab158 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:03.882940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:03.923626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd9747-1e7e-442b-ada2-6d772a3ab158 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:03.957196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c2f76-d476-4604-9b88-6f54c729a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:03.957816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c2f76-d476-4604-9b88-6f54c729a20a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:03.958096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c2f76-d476-4604-9b88-6f54c729a20a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:03.958370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c2f76-d476-4604-9b88-6f54c729a20a None admin] PATH_INFO: `/v3/domains/393f874ac1e94d1b8b9cfaa6acb1b86d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:03.968504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c2f76-d476-4604-9b88-6f54c729a20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=393f874ac1e94d1b8b9cfaa6acb1b86d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:03.970004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c2f76-d476-4604-9b88-6f54c729a20a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:03.971583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 587/1172] 10.222.0.50 () {56 vars in 1183 bytes} [Thu Sep 15 10:45:03 2022] GET /identity/v3/domains/393f874ac1e94d1b8b9cfaa6acb1b86d => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:03.977518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdf687-bf07-4b63-8fa6-990878a98c09 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:03.979674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:04.076458 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdf687-bf07-4b63-8fa6-990878a98c09 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:04.109380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2b776-9cab-4dcc-9a17-21f04fc8b24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:04.109974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b776-9cab-4dcc-9a17-21f04fc8b24b None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:04.110256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b776-9cab-4dcc-9a17-21f04fc8b24b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:04.110528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b776-9cab-4dcc-9a17-21f04fc8b24b None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:04.111169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2b776-9cab-4dcc-9a17-21f04fc8b24b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:04.112594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2b776-9cab-4dcc-9a17-21f04fc8b24b None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:04.138617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 586/1173] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:45:03 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:45:06.309122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d15e2a-c211-44a1-a8ac-7555084a3490 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:06.309663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d15e2a-c211-44a1-a8ac-7555084a3490 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:06.309663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d15e2a-c211-44a1-a8ac-7555084a3490 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:06.311027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 588/1174] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:45:06.316424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a99f9b-05c0-4817-a5d7-fc95d0456c19 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:06.316880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a99f9b-05c0-4817-a5d7-fc95d0456c19 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:06.317004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a99f9b-05c0-4817-a5d7-fc95d0456c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:06.336264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e7a99f9b-05c0-4817-a5d7-fc95d0456c19 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:45:06.337472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e7a99f9b-05c0-4817-a5d7-fc95d0456c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:45:06.385090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:06.403810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 587/1175] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:45:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:45:06.677348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d15e2a-c211-44a1-a8ac-7555084a3490 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:06.778865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d15e2a-c211-44a1-a8ac-7555084a3490 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:06.796100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:06.812921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce573902-0111-4b3b-91ad-a05c82315ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:06.813556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce573902-0111-4b3b-91ad-a05c82315ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:06.813840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce573902-0111-4b3b-91ad-a05c82315ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:06.814114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce573902-0111-4b3b-91ad-a05c82315ad7 None admin] PATH_INFO: `/v3/domains/393f874ac1e94d1b8b9cfaa6acb1b86d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:06.815240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce573902-0111-4b3b-91ad-a05c82315ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=393f874ac1e94d1b8b9cfaa6acb1b86d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:06.816691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce573902-0111-4b3b-91ad-a05c82315ad7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:06.818297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 589/1176] 10.222.0.50 () {56 vars in 1183 bytes} [Thu Sep 15 10:45:06 2022] GET /identity/v3/domains/393f874ac1e94d1b8b9cfaa6acb1b86d => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:06.824438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a99f9b-05c0-4817-a5d7-fc95d0456c19 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:06.923642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a99f9b-05c0-4817-a5d7-fc95d0456c19 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:06.956707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9a323-b1dd-41da-be1f-409647fd11fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:06.957257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9a323-b1dd-41da-be1f-409647fd11fb None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:06.957532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9a323-b1dd-41da-be1f-409647fd11fb None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:06.957799 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9a323-b1dd-41da-be1f-409647fd11fb None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:06.958436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9a323-b1dd-41da-be1f-409647fd11fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:06.959860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9a323-b1dd-41da-be1f-409647fd11fb None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:06.981931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 588/1177] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:45:06 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:45:09.174618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b67c2-cbe3-4c98-affd-ff5cd6242ead None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:09.175155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b67c2-cbe3-4c98-affd-ff5cd6242ead None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:09.175155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b67c2-cbe3-4c98-affd-ff5cd6242ead None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:09.176497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 590/1178] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:45:09.182444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438dc7a-205b-4970-9f2f-f0b80c1519b3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:09.182848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438dc7a-205b-4970-9f2f-f0b80c1519b3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:09.182996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438dc7a-205b-4970-9f2f-f0b80c1519b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:09.201920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4438dc7a-205b-4970-9f2f-f0b80c1519b3 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:45:09.203181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4438dc7a-205b-4970-9f2f-f0b80c1519b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:45:09.236690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:09.270450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 589/1179] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:45:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:45:09.539990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b67c2-cbe3-4c98-affd-ff5cd6242ead None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:09.640638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b67c2-cbe3-4c98-affd-ff5cd6242ead None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:09.673101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b0dfa-423a-49d1-bc84-7a7b6f1e54a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:09.673716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b0dfa-423a-49d1-bc84-7a7b6f1e54a3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:09.673996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b0dfa-423a-49d1-bc84-7a7b6f1e54a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:09.674265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b0dfa-423a-49d1-bc84-7a7b6f1e54a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:09.684611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b0dfa-423a-49d1-bc84-7a7b6f1e54a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-d07b0dfa-423a-49d1-bc84-7a7b6f1e54a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:45:09.689352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:45:09.690476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:45:09.691592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 591/1180] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:45:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:45:09.697284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438dc7a-205b-4970-9f2f-f0b80c1519b3 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:09.794274 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438dc7a-205b-4970-9f2f-f0b80c1519b3 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:09.828260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be7ef-2d3b-467d-87ce-545590eb2bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:09.828885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be7ef-2d3b-467d-87ce-545590eb2bae None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:09.829166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be7ef-2d3b-467d-87ce-545590eb2bae None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:09.829436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be7ef-2d3b-467d-87ce-545590eb2bae None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:09.830092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0be7ef-2d3b-467d-87ce-545590eb2bae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:45:09.830394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0be7ef-2d3b-467d-87ce-545590eb2bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:09.831794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0be7ef-2d3b-467d-87ce-545590eb2bae None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:09.840137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 590/1181] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:45:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:09.846034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b0dfa-423a-49d1-bc84-7a7b6f1e54a3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:09.892901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:09.945393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b0dfa-423a-49d1-bc84-7a7b6f1e54a3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:09.979801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52877250-a1e2-4144-9d99-2cd79a6769fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:09.980411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52877250-a1e2-4144-9d99-2cd79a6769fd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:09.980690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52877250-a1e2-4144-9d99-2cd79a6769fd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:09.980996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52877250-a1e2-4144-9d99-2cd79a6769fd None admin] PATH_INFO: `/v3/users/9e909ce2dec747b4900a40af411b1b7c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:09.994670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52877250-a1e2-4144-9d99-2cd79a6769fd None admin] RBAC: Authorizing `identity:get_user(user_id=9e909ce2dec747b4900a40af411b1b7c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:09.996110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52877250-a1e2-4144-9d99-2cd79a6769fd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:09.997725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 592/1182] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:45:09 2022] GET /identity/v3/users/9e909ce2dec747b4900a40af411b1b7c => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:10.003502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be7ef-2d3b-467d-87ce-545590eb2bae None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:10.035494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:10.104119 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be7ef-2d3b-467d-87ce-545590eb2bae None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:10.136643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23358ec6-b30c-415d-bcb9-dc1349be07ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:10.137276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23358ec6-b30c-415d-bcb9-dc1349be07ca None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:10.137554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23358ec6-b30c-415d-bcb9-dc1349be07ca None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:10.137826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23358ec6-b30c-415d-bcb9-dc1349be07ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:10.204858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23358ec6-b30c-415d-bcb9-dc1349be07ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:10.206354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23358ec6-b30c-415d-bcb9-dc1349be07ca None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:10.235253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:10.302732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 591/1183] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:45:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:45:10.309012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52877250-a1e2-4144-9d99-2cd79a6769fd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:10.407220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52877250-a1e2-4144-9d99-2cd79a6769fd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:10.409505 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:10.442038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a09e2-7493-4d2d-8bf6-0642bdc95115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:10.442663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a09e2-7493-4d2d-8bf6-0642bdc95115 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:10.442945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a09e2-7493-4d2d-8bf6-0642bdc95115 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:10.443217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a09e2-7493-4d2d-8bf6-0642bdc95115 None admin] PATH_INFO: `/v3/projects/488a30079dcd4390b57fa4570e80dc50` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:10.444969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a09e2-7493-4d2d-8bf6-0642bdc95115 None admin] RBAC: Authorizing `identity:get_project(project_id=488a30079dcd4390b57fa4570e80dc50)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:10.446417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a09e2-7493-4d2d-8bf6-0642bdc95115 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:10.448290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 593/1184] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:45:10 2022] GET /identity/v3/projects/488a30079dcd4390b57fa4570e80dc50 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:10.454371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23358ec6-b30c-415d-bcb9-dc1349be07ca None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:10.550996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23358ec6-b30c-415d-bcb9-dc1349be07ca None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:10.581946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:10.587329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047ba89-871c-44f2-b2c7-e915ddb5f71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:10.587923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047ba89-871c-44f2-b2c7-e915ddb5f71a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:10.588201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047ba89-871c-44f2-b2c7-e915ddb5f71a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:10.588468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047ba89-871c-44f2-b2c7-e915ddb5f71a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:10.589209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047ba89-871c-44f2-b2c7-e915ddb5f71a None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=488a30079dcd4390b57fa4570e80dc50, user.id=9e909ce2dec747b4900a40af411b1b7c) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:45:10.589514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047ba89-871c-44f2-b2c7-e915ddb5f71a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:10.590932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047ba89-871c-44f2-b2c7-e915ddb5f71a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:10.596471 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 592/1185] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:45:10 2022] GET /identity/v3/role_assignments?user.id=9e909ce2dec747b4900a40af411b1b7c&scope.project.id=488a30079dcd4390b57fa4570e80dc50&role.id=0be9777002c641b4a47f78df647cc173 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:12.873506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2347bb-29f2-4eda-88d2-0392bad80a3c None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:12.874464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2347bb-29f2-4eda-88d2-0392bad80a3c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:12.874464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2347bb-29f2-4eda-88d2-0392bad80a3c None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:12.875265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 594/1186] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:45:12.880517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec94e4-3568-4928-8aa2-ed6f55496dde None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:12.880921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec94e4-3568-4928-8aa2-ed6f55496dde None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:12.881112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec94e4-3568-4928-8aa2-ed6f55496dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:12.900239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-83ec94e4-3568-4928-8aa2-ed6f55496dde None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:45:12.901456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-83ec94e4-3568-4928-8aa2-ed6f55496dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:45:12.969733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 593/1187] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:45:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:45:13.238906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2347bb-29f2-4eda-88d2-0392bad80a3c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:13.342685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2347bb-29f2-4eda-88d2-0392bad80a3c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:13.376262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070fa1b-9bb4-4a04-b81d-723c6fb6bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:13.376908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070fa1b-9bb4-4a04-b81d-723c6fb6bc51 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:13.377190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070fa1b-9bb4-4a04-b81d-723c6fb6bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:13.377462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070fa1b-9bb4-4a04-b81d-723c6fb6bc51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:13.387848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070fa1b-9bb4-4a04-b81d-723c6fb6bc51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6070fa1b-9bb4-4a04-b81d-723c6fb6bc51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:45:13.392623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:45:13.393865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:45:13.395157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 595/1188] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:45:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:45:13.399981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec94e4-3568-4928-8aa2-ed6f55496dde None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:13.498074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec94e4-3568-4928-8aa2-ed6f55496dde None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:13.523409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:13.532173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ec90d-94ed-469c-aae0-2e68549e57d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:13.532777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ec90d-94ed-469c-aae0-2e68549e57d6 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:13.533085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ec90d-94ed-469c-aae0-2e68549e57d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:13.533360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ec90d-94ed-469c-aae0-2e68549e57d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:13.534015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ec90d-94ed-469c-aae0-2e68549e57d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:45:13.534317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ec90d-94ed-469c-aae0-2e68549e57d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:13.535693 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ec90d-94ed-469c-aae0-2e68549e57d6 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:13.546524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 594/1189] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:45:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:13.552304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070fa1b-9bb4-4a04-b81d-723c6fb6bc51 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:13.584771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:13.652560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070fa1b-9bb4-4a04-b81d-723c6fb6bc51 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:13.685913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad465cf3-b743-49c6-acd8-7411ff3632c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:13.686532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad465cf3-b743-49c6-acd8-7411ff3632c9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:13.686812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad465cf3-b743-49c6-acd8-7411ff3632c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:13.687086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad465cf3-b743-49c6-acd8-7411ff3632c9 None admin] PATH_INFO: `/v3/users/9e909ce2dec747b4900a40af411b1b7c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:13.688190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad465cf3-b743-49c6-acd8-7411ff3632c9 None admin] RBAC: Authorizing `identity:get_user(user_id=9e909ce2dec747b4900a40af411b1b7c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:13.689661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad465cf3-b743-49c6-acd8-7411ff3632c9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:13.691231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 596/1190] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:45:13 2022] GET /identity/v3/users/9e909ce2dec747b4900a40af411b1b7c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:13.697772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ec90d-94ed-469c-aae0-2e68549e57d6 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:13.795322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ec90d-94ed-469c-aae0-2e68549e57d6 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:13.827973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac220717-89db-4dad-b1d1-720c3c5b2d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:13.828593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac220717-89db-4dad-b1d1-720c3c5b2d88 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:13.828901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac220717-89db-4dad-b1d1-720c3c5b2d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:13.829176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac220717-89db-4dad-b1d1-720c3c5b2d88 None admin] PATH_INFO: `/v3/projects/488a30079dcd4390b57fa4570e80dc50` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:13.830279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac220717-89db-4dad-b1d1-720c3c5b2d88 None admin] RBAC: Authorizing `identity:get_project(project_id=488a30079dcd4390b57fa4570e80dc50)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:13.831709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac220717-89db-4dad-b1d1-720c3c5b2d88 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:13.833358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 595/1191] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:45:13 2022] GET /identity/v3/projects/488a30079dcd4390b57fa4570e80dc50 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:13.839259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad465cf3-b743-49c6-acd8-7411ff3632c9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:13.934690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:13.937740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad465cf3-b743-49c6-acd8-7411ff3632c9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:13.971498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277b714-a820-4078-8755-a9e26c7a2b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:13.972144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277b714-a820-4078-8755-a9e26c7a2b93 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:13.972425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277b714-a820-4078-8755-a9e26c7a2b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:13.972697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277b714-a820-4078-8755-a9e26c7a2b93 None admin] PATH_INFO: `/v3/projects/488a30079dcd4390b57fa4570e80dc50/users/9e909ce2dec747b4900a40af411b1b7c/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:13.974603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277b714-a820-4078-8755-a9e26c7a2b93 None admin] RBAC: Authorizing `identity:create_grant(project_id=488a30079dcd4390b57fa4570e80dc50, user_id=9e909ce2dec747b4900a40af411b1b7c, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:13.976640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277b714-a820-4078-8755-a9e26c7a2b93 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:13.990957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 597/1192] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:45:13 2022] PUT /identity/v3/projects/488a30079dcd4390b57fa4570e80dc50/users/9e909ce2dec747b4900a40af411b1b7c/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:45:16.123701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50459440-b301-4b06-bb25-293a457dde4f None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:16.124333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50459440-b301-4b06-bb25-293a457dde4f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:16.124333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50459440-b301-4b06-bb25-293a457dde4f None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:16.125747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 596/1193] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:45:16.131080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a28c21-c860-41fb-abaa-0fb0d3701e6d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:16.132090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a28c21-c860-41fb-abaa-0fb0d3701e6d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:16.132329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a28c21-c860-41fb-abaa-0fb0d3701e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:16.150971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b7a28c21-c860-41fb-abaa-0fb0d3701e6d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:45:16.152179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b7a28c21-c860-41fb-abaa-0fb0d3701e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:45:16.218741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 598/1194] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:45:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:45:16.489142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50459440-b301-4b06-bb25-293a457dde4f None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:16.549669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:16.595950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50459440-b301-4b06-bb25-293a457dde4f None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:16.628258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ea34d-8b41-4da9-8833-d53a8c28589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:16.628912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ea34d-8b41-4da9-8833-d53a8c28589b None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:16.629194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ea34d-8b41-4da9-8833-d53a8c28589b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:16.629465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ea34d-8b41-4da9-8833-d53a8c28589b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:16.639894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ea34d-8b41-4da9-8833-d53a8c28589b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-130ea34d-8b41-4da9-8833-d53a8c28589b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:45:16.644744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:45:16.645930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:45:16.647431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 597/1195] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:45:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:45:16.652465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a28c21-c860-41fb-abaa-0fb0d3701e6d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:16.741239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:16.751839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a28c21-c860-41fb-abaa-0fb0d3701e6d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:16.785368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5742069-4790-4565-85ae-d6c26bfb3931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:16.785974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5742069-4790-4565-85ae-d6c26bfb3931 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:16.786257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5742069-4790-4565-85ae-d6c26bfb3931 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:16.786530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5742069-4790-4565-85ae-d6c26bfb3931 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:16.787185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5742069-4790-4565-85ae-d6c26bfb3931 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:45:16.788010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5742069-4790-4565-85ae-d6c26bfb3931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:16.789439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5742069-4790-4565-85ae-d6c26bfb3931 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:16.797278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 599/1196] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:45:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:16.803746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ea34d-8b41-4da9-8833-d53a8c28589b None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:16.900685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ea34d-8b41-4da9-8833-d53a8c28589b None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:16.916782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:16.935037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac74dd-f601-4d07-a140-de56446f0ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:16.935649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac74dd-f601-4d07-a140-de56446f0ceb None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:16.935926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac74dd-f601-4d07-a140-de56446f0ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:16.936197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac74dd-f601-4d07-a140-de56446f0ceb None admin] PATH_INFO: `/v3/users/9e909ce2dec747b4900a40af411b1b7c` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:16.937781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac74dd-f601-4d07-a140-de56446f0ceb None admin] RBAC: Authorizing `identity:get_user(user_id=9e909ce2dec747b4900a40af411b1b7c)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:16.939206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac74dd-f601-4d07-a140-de56446f0ceb None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:16.940768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 598/1197] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Sep 15 10:45:16 2022] GET /identity/v3/users/9e909ce2dec747b4900a40af411b1b7c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:16.946598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5742069-4790-4565-85ae-d6c26bfb3931 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:17.045034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5742069-4790-4565-85ae-d6c26bfb3931 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:17.078083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1340c9-4524-446a-9d18-b72fc0b19ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:17.078688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1340c9-4524-446a-9d18-b72fc0b19ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:17.078972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1340c9-4524-446a-9d18-b72fc0b19ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:17.079248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1340c9-4524-446a-9d18-b72fc0b19ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:17.101078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:17.146231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1340c9-4524-446a-9d18-b72fc0b19ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:17.148364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1340c9-4524-446a-9d18-b72fc0b19ee6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:17.244164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 600/1198] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:45:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:45:17.250252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac74dd-f601-4d07-a140-de56446f0ceb None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:17.348683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac74dd-f601-4d07-a140-de56446f0ceb None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:17.384574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f425ed-9280-4853-87df-a3c28776d13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:17.385219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f425ed-9280-4853-87df-a3c28776d13a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:17.385497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f425ed-9280-4853-87df-a3c28776d13a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:17.385766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f425ed-9280-4853-87df-a3c28776d13a None admin] PATH_INFO: `/v3/projects/488a30079dcd4390b57fa4570e80dc50` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:17.386872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f425ed-9280-4853-87df-a3c28776d13a None admin] RBAC: Authorizing `identity:get_project(project_id=488a30079dcd4390b57fa4570e80dc50)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:17.388286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f425ed-9280-4853-87df-a3c28776d13a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:17.389910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 599/1199] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Sep 15 10:45:17 2022] GET /identity/v3/projects/488a30079dcd4390b57fa4570e80dc50 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:17.396012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1340c9-4524-446a-9d18-b72fc0b19ee6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:45:17.448070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:17.497569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1340c9-4524-446a-9d18-b72fc0b19ee6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:45:17.531955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89c1f5-0d44-433b-b4cf-02075db475af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:45:17.532553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c1f5-0d44-433b-b4cf-02075db475af None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:45:17.532862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c1f5-0d44-433b-b4cf-02075db475af None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:45:17.533136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c1f5-0d44-433b-b4cf-02075db475af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:45:17.533850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89c1f5-0d44-433b-b4cf-02075db475af None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=488a30079dcd4390b57fa4570e80dc50, user.id=9e909ce2dec747b4900a40af411b1b7c) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:45:17.534155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89c1f5-0d44-433b-b4cf-02075db475af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:45:17.535594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89c1f5-0d44-433b-b4cf-02075db475af None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:45:17.542608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 601/1200] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:45:17 2022] GET /identity/v3/role_assignments?user.id=9e909ce2dec747b4900a40af411b1b7c&scope.project.id=488a30079dcd4390b57fa4570e80dc50&role.id=0be9777002c641b4a47f78df647cc173 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:45:27.378782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=74 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:45:27.539727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=42 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:13.330420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c8125-7a45-4281-99eb-29ff7f383983 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:13.331336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c8125-7a45-4281-99eb-29ff7f383983 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:13.331560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c8125-7a45-4281-99eb-29ff7f383983 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:13.332985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 600/1201] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:13.338590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf60c13-9835-485b-bf52-60be4639e825 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:13.339046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf60c13-9835-485b-bf52-60be4639e825 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:13.339319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf60c13-9835-485b-bf52-60be4639e825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:13.360395 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4cf60c13-9835-485b-bf52-60be4639e825 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:13.361643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4cf60c13-9835-485b-bf52-60be4639e825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:13.461011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 602/1202] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:13.785568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c8125-7a45-4281-99eb-29ff7f383983 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:13.916638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c8125-7a45-4281-99eb-29ff7f383983 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:13.963930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2117-ec00-4c1b-a043-f908e098e41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:13.965047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2117-ec00-4c1b-a043-f908e098e41d None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:13.965635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2117-ec00-4c1b-a043-f908e098e41d None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:13.966160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2117-ec00-4c1b-a043-f908e098e41d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:13.979449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af2117-ec00-4c1b-a043-f908e098e41d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:13.981449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af2117-ec00-4c1b-a043-f908e098e41d None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-70af2117-ec00-4c1b-a043-f908e098e41d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:14.000248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:14.001505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:14.002550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:14.002550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:14.002550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:14.004966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 601/1203] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:14.018602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf60c13-9835-485b-bf52-60be4639e825 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:14.156737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf60c13-9835-485b-bf52-60be4639e825 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:14.195846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:14.202785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3efb5-66e5-4e14-a46b-cbf3fe970c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:14.203545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3efb5-66e5-4e14-a46b-cbf3fe970c89 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:14.203867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3efb5-66e5-4e14-a46b-cbf3fe970c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:14.204101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3efb5-66e5-4e14-a46b-cbf3fe970c89 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:14.204746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3efb5-66e5-4e14-a46b-cbf3fe970c89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:14.205091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3efb5-66e5-4e14-a46b-cbf3fe970c89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:14.207060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3efb5-66e5-4e14-a46b-cbf3fe970c89 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:14.224421 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 603/1204] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:14.230320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2117-ec00-4c1b-a043-f908e098e41d None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:14.302671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:14.359993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2117-ec00-4c1b-a043-f908e098e41d None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:14.401010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d47c5f-bdfd-42c6-a3af-621d16188f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:14.401714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d47c5f-bdfd-42c6-a3af-621d16188f2b None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:14.401994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d47c5f-bdfd-42c6-a3af-621d16188f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:14.402305 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d47c5f-bdfd-42c6-a3af-621d16188f2b None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:14.403013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d47c5f-bdfd-42c6-a3af-621d16188f2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:14.404545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d47c5f-bdfd-42c6-a3af-621d16188f2b None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:14.431196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 602/1205] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Sep 15 10:46:14 2022] POST /identity/v3/users => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:46:16.889855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27bc8c-558b-4a14-9783-00c065ec6b2d None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:16.889855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27bc8c-558b-4a14-9783-00c065ec6b2d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:16.889855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27bc8c-558b-4a14-9783-00c065ec6b2d None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:16.890531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 604/1206] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:16.897653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e26ad-682b-467d-a5f2-96a4778d93c2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:16.898102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e26ad-682b-467d-a5f2-96a4778d93c2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:16.898200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e26ad-682b-467d-a5f2-96a4778d93c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:16.919659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-904e26ad-682b-467d-a5f2-96a4778d93c2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:16.921328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-904e26ad-682b-467d-a5f2-96a4778d93c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:17.019002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 603/1207] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 10:46:17.321970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27bc8c-558b-4a14-9783-00c065ec6b2d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:17.453544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27bc8c-558b-4a14-9783-00c065ec6b2d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:17.494053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919724e-fed8-4ea0-b345-32d74207cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:17.494702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919724e-fed8-4ea0-b345-32d74207cd08 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:17.494979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919724e-fed8-4ea0-b345-32d74207cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:17.495250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919724e-fed8-4ea0-b345-32d74207cd08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:17.506598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919724e-fed8-4ea0-b345-32d74207cd08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6919724e-fed8-4ea0-b345-32d74207cd08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:46:17.510964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:17.512245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:17.513382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 605/1208] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:46:17.518424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e26ad-682b-467d-a5f2-96a4778d93c2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:17.573924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:17.657242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e26ad-682b-467d-a5f2-96a4778d93c2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:17.701186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da845480-850d-44b8-a500-cf8ee50905bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:17.701869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da845480-850d-44b8-a500-cf8ee50905bc None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:17.702193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da845480-850d-44b8-a500-cf8ee50905bc None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:17.702539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da845480-850d-44b8-a500-cf8ee50905bc None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:17.703295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da845480-850d-44b8-a500-cf8ee50905bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:17.703639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da845480-850d-44b8-a500-cf8ee50905bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:17.705191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da845480-850d-44b8-a500-cf8ee50905bc None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:17.716262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 604/1209] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:17.723362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919724e-fed8-4ea0-b345-32d74207cd08 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:17.782502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:17.845304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919724e-fed8-4ea0-b345-32d74207cd08 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:17.888542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aaba1-ed5e-4364-ab2e-23f768d62903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:17.890834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aaba1-ed5e-4364-ab2e-23f768d62903 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:17.890834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aaba1-ed5e-4364-ab2e-23f768d62903 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:17.890834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aaba1-ed5e-4364-ab2e-23f768d62903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:17.901455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99aaba1-ed5e-4364-ab2e-23f768d62903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:17.902924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99aaba1-ed5e-4364-ab2e-23f768d62903 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-f99aaba1-ed5e-4364-ab2e-23f768d62903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:17.919015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:17.920297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:17.921352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:17.921352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:17.921352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:17.921352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 606/1210] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:17.926788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da845480-850d-44b8-a500-cf8ee50905bc None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:18.052171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:18.055548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da845480-850d-44b8-a500-cf8ee50905bc None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:18.097231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425c9a9-20ba-40ad-9c46-6d3e178a3d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:18.097908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425c9a9-20ba-40ad-9c46-6d3e178a3d15 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:18.098188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425c9a9-20ba-40ad-9c46-6d3e178a3d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:18.098461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425c9a9-20ba-40ad-9c46-6d3e178a3d15 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:18.099110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425c9a9-20ba-40ad-9c46-6d3e178a3d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:18.099413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425c9a9-20ba-40ad-9c46-6d3e178a3d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:18.100813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425c9a9-20ba-40ad-9c46-6d3e178a3d15 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:18.114855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 605/1211] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:18.120374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aaba1-ed5e-4364-ab2e-23f768d62903 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:18.231104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aaba1-ed5e-4364-ab2e-23f768d62903 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:18.232782 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:18.277425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821ee9d-54ab-4d65-8fdf-b9915c4994dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:18.278289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821ee9d-54ab-4d65-8fdf-b9915c4994dc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:18.278606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821ee9d-54ab-4d65-8fdf-b9915c4994dc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:18.278911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821ee9d-54ab-4d65-8fdf-b9915c4994dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:18.287158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821ee9d-54ab-4d65-8fdf-b9915c4994dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:18.289045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821ee9d-54ab-4d65-8fdf-b9915c4994dc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0821ee9d-54ab-4d65-8fdf-b9915c4994dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:46:18.305058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:18.306099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:18.312378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 607/1212] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:18.313183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425c9a9-20ba-40ad-9c46-6d3e178a3d15 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:18.423265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425c9a9-20ba-40ad-9c46-6d3e178a3d15 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:18.465641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28afcf-638e-4236-8820-2d44e4f83a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:18.466247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28afcf-638e-4236-8820-2d44e4f83a20 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:18.466529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28afcf-638e-4236-8820-2d44e4f83a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:18.466804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28afcf-638e-4236-8820-2d44e4f83a20 None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:18.467637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28afcf-638e-4236-8820-2d44e4f83a20 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:18.467946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28afcf-638e-4236-8820-2d44e4f83a20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:18.469383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28afcf-638e-4236-8820-2d44e4f83a20 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:18.490114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 606/1213] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Sep 15 10:46:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:18.495698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821ee9d-54ab-4d65-8fdf-b9915c4994dc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:18.626560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821ee9d-54ab-4d65-8fdf-b9915c4994dc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:18.667764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6316f0-a7d0-4e9c-9a2c-3c4c96e998d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:18.668637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6316f0-a7d0-4e9c-9a2c-3c4c96e998d4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:18.668956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6316f0-a7d0-4e9c-9a2c-3c4c96e998d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:18.669232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6316f0-a7d0-4e9c-9a2c-3c4c96e998d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:18.705853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:18.750303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6316f0-a7d0-4e9c-9a2c-3c4c96e998d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:18.751783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6316f0-a7d0-4e9c-9a2c-3c4c96e998d4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:18.862178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 608/1214] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:46:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:46:18.867920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28afcf-638e-4236-8820-2d44e4f83a20 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:18.892223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:19.009141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28afcf-638e-4236-8820-2d44e4f83a20 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:19.057788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5eb06-e923-4e99-9d61-a49ac6cacaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:19.058837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5eb06-e923-4e99-9d61-a49ac6cacaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:19.059272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5eb06-e923-4e99-9d61-a49ac6cacaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:19.059698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5eb06-e923-4e99-9d61-a49ac6cacaf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:19.072481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5eb06-e923-4e99-9d61-a49ac6cacaf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:19.074353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5eb06-e923-4e99-9d61-a49ac6cacaf1 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-71c5eb06-e923-4e99-9d61-a49ac6cacaf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:19.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:19.094821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:19.098093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:19.098093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:19.098093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:19.098093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 607/1215] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:19.101939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6316f0-a7d0-4e9c-9a2c-3c4c96e998d4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:19.183732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:19.258701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6316f0-a7d0-4e9c-9a2c-3c4c96e998d4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:19.317212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6672c8a-7200-4172-ab44-0c36e2587ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:19.318171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6672c8a-7200-4172-ab44-0c36e2587ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:19.321163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6672c8a-7200-4172-ab44-0c36e2587ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:19.321455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6672c8a-7200-4172-ab44-0c36e2587ce1 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:19.322111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6672c8a-7200-4172-ab44-0c36e2587ce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:19.322442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6672c8a-7200-4172-ab44-0c36e2587ce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:19.323852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6672c8a-7200-4172-ab44-0c36e2587ce1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:19.339606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 609/1216] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:19.346388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5eb06-e923-4e99-9d61-a49ac6cacaf1 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:19.423228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:19.456758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5eb06-e923-4e99-9d61-a49ac6cacaf1 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:19.494201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48356-3589-48c6-8a98-e97307080508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:19.495075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48356-3589-48c6-8a98-e97307080508 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:19.495356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48356-3589-48c6-8a98-e97307080508 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:19.495666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48356-3589-48c6-8a98-e97307080508 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:19.502938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec48356-3589-48c6-8a98-e97307080508 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:19.504398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec48356-3589-48c6-8a98-e97307080508 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-cec48356-3589-48c6-8a98-e97307080508 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:46:19.516509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:19.518544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:19.519889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:46:19.519889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:46:19.519889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:46:19.519889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:46:19.519889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:46:19.519889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:46:19.519889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:19.519889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:19.519889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 608/1217] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:19.524341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6672c8a-7200-4172-ab44-0c36e2587ce1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:19.638088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6672c8a-7200-4172-ab44-0c36e2587ce1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:19.660355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:19.671655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a766491-683e-429c-8d63-09ed86837567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:19.672265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a766491-683e-429c-8d63-09ed86837567 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:19.672545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a766491-683e-429c-8d63-09ed86837567 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:19.672841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a766491-683e-429c-8d63-09ed86837567 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:19.673531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a766491-683e-429c-8d63-09ed86837567 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:19.674385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a766491-683e-429c-8d63-09ed86837567 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:19.675887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a766491-683e-429c-8d63-09ed86837567 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:19.688010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 610/1218] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:19.695249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48356-3589-48c6-8a98-e97307080508 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:19.817526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48356-3589-48c6-8a98-e97307080508 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:19.848746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:19.855243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9056fc-ea0b-41ef-8b3a-5f218eee8492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:19.855831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9056fc-ea0b-41ef-8b3a-5f218eee8492 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:19.856114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9056fc-ea0b-41ef-8b3a-5f218eee8492 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:19.856386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9056fc-ea0b-41ef-8b3a-5f218eee8492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:19.857155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9056fc-ea0b-41ef-8b3a-5f218eee8492 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=0620b2af88034caa8de99e9b48b05f7e) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:19.857466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9056fc-ea0b-41ef-8b3a-5f218eee8492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:19.859844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9056fc-ea0b-41ef-8b3a-5f218eee8492 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:19.866002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 609/1219] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:46:19 2022] GET /identity/v3/role_assignments?user.id=0620b2af88034caa8de99e9b48b05f7e&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:46:21.997850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ebaad-6458-431b-ac58-8c1de5281a9a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:21.998420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ebaad-6458-431b-ac58-8c1de5281a9a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:21.998648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ebaad-6458-431b-ac58-8c1de5281a9a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:22.000585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 611/1220] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:22.006205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318455a-ddaa-4044-b14d-f5611d3ebc05 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:22.006674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318455a-ddaa-4044-b14d-f5611d3ebc05 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:22.006778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318455a-ddaa-4044-b14d-f5611d3ebc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:22.026786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-b318455a-ddaa-4044-b14d-f5611d3ebc05 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:22.027982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b318455a-ddaa-4044-b14d-f5611d3ebc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:22.107970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 610/1221] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:22.386405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ebaad-6458-431b-ac58-8c1de5281a9a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:22.499734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ebaad-6458-431b-ac58-8c1de5281a9a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:22.537742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a9117-e9e2-4c31-9a05-a048d03f0af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:22.540456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a9117-e9e2-4c31-9a05-a048d03f0af4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:22.540779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a9117-e9e2-4c31-9a05-a048d03f0af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:22.541042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a9117-e9e2-4c31-9a05-a048d03f0af4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:22.552714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a9117-e9e2-4c31-9a05-a048d03f0af4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-268a9117-e9e2-4c31-9a05-a048d03f0af4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:46:22.557974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:22.559357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:22.560882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 612/1222] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:46:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:22.566332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318455a-ddaa-4044-b14d-f5611d3ebc05 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:22.674675 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318455a-ddaa-4044-b14d-f5611d3ebc05 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:22.699469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:22.712330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fba7ac-e4e1-4d2a-8e9a-71aeeb48293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:22.712973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fba7ac-e4e1-4d2a-8e9a-71aeeb48293c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:22.713338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fba7ac-e4e1-4d2a-8e9a-71aeeb48293c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:22.713616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fba7ac-e4e1-4d2a-8e9a-71aeeb48293c None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:22.714422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fba7ac-e4e1-4d2a-8e9a-71aeeb48293c None admin] RBAC: Adding query filter params (name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:22.714729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fba7ac-e4e1-4d2a-8e9a-71aeeb48293c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:22.716248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fba7ac-e4e1-4d2a-8e9a-71aeeb48293c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:22.726260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 611/1223] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:46:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:22.732276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a9117-e9e2-4c31-9a05-a048d03f0af4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:22.781823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:22.842117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a9117-e9e2-4c31-9a05-a048d03f0af4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:22.881016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815dc66-ade6-4727-9d05-88d5a2e98d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:22.881656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815dc66-ade6-4727-9d05-88d5a2e98d6e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:22.881941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815dc66-ade6-4727-9d05-88d5a2e98d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:22.882212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815dc66-ade6-4727-9d05-88d5a2e98d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:22.893221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815dc66-ade6-4727-9d05-88d5a2e98d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:22.894714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815dc66-ade6-4727-9d05-88d5a2e98d6e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2815dc66-ade6-4727-9d05-88d5a2e98d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:22.910921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:22.912187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:22.913538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:22.913538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:22.913538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:22.913538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 613/1224] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:22.918316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fba7ac-e4e1-4d2a-8e9a-71aeeb48293c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:23.036388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fba7ac-e4e1-4d2a-8e9a-71aeeb48293c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:23.073845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bea62-3a5d-40eb-a73b-d4a3d86beb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:23.074457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bea62-3a5d-40eb-a73b-d4a3d86beb6f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:23.074737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bea62-3a5d-40eb-a73b-d4a3d86beb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:23.075042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bea62-3a5d-40eb-a73b-d4a3d86beb6f None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:23.075670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bea62-3a5d-40eb-a73b-d4a3d86beb6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:23.076669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bea62-3a5d-40eb-a73b-d4a3d86beb6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:23.078220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bea62-3a5d-40eb-a73b-d4a3d86beb6f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:23.092716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 612/1225] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:46:23.098693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815dc66-ade6-4727-9d05-88d5a2e98d6e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:23.202513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:23.210196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815dc66-ade6-4727-9d05-88d5a2e98d6e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:23.263762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612c64a-60a4-4447-9034-9491651353b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:23.264565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612c64a-60a4-4447-9034-9491651353b8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:23.264898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612c64a-60a4-4447-9034-9491651353b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:23.265182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612c64a-60a4-4447-9034-9491651353b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:23.273734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612c64a-60a4-4447-9034-9491651353b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:23.275187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612c64a-60a4-4447-9034-9491651353b8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e612c64a-60a4-4447-9034-9491651353b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:46:23.288120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:23.291055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:23.292283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 614/1226] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:23.295050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bea62-3a5d-40eb-a73b-d4a3d86beb6f None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:23.348646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:23.404821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bea62-3a5d-40eb-a73b-d4a3d86beb6f None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:23.447192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b93c7-654c-4c38-addf-5d0f80f083be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:23.447801 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b93c7-654c-4c38-addf-5d0f80f083be None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:23.448102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b93c7-654c-4c38-addf-5d0f80f083be None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:23.448355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b93c7-654c-4c38-addf-5d0f80f083be None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:23.449112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b93c7-654c-4c38-addf-5d0f80f083be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:23.449416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b93c7-654c-4c38-addf-5d0f80f083be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:23.450845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b93c7-654c-4c38-addf-5d0f80f083be None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:23.468765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 613/1227] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Sep 15 10:46:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:23.475393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612c64a-60a4-4447-9034-9491651353b8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:23.586083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612c64a-60a4-4447-9034-9491651353b8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:23.620647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cd1bb-a0ff-450f-b8c9-6583b47ac351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:23.621329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cd1bb-a0ff-450f-b8c9-6583b47ac351 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:23.621615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cd1bb-a0ff-450f-b8c9-6583b47ac351 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:23.621891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cd1bb-a0ff-450f-b8c9-6583b47ac351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:23.634727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cd1bb-a0ff-450f-b8c9-6583b47ac351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:23.636295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cd1bb-a0ff-450f-b8c9-6583b47ac351 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-7c0cd1bb-a0ff-450f-b8c9-6583b47ac351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:23.652748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:23.654140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:23.655091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:23.655091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:23.655091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:23.655091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 615/1228] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:23.659909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b93c7-654c-4c38-addf-5d0f80f083be None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:23.770407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b93c7-654c-4c38-addf-5d0f80f083be None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:23.772694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:23.805035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942b0c9-5575-417b-925c-2373fd6aad2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:23.805637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942b0c9-5575-417b-925c-2373fd6aad2a None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:23.805934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942b0c9-5575-417b-925c-2373fd6aad2a None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:23.806183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942b0c9-5575-417b-925c-2373fd6aad2a None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:23.806818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942b0c9-5575-417b-925c-2373fd6aad2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:23.807166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942b0c9-5575-417b-925c-2373fd6aad2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:23.808482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942b0c9-5575-417b-925c-2373fd6aad2a None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:23.820993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 614/1229] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:23.826841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cd1bb-a0ff-450f-b8c9-6583b47ac351 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:23.837132 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:23.936622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cd1bb-a0ff-450f-b8c9-6583b47ac351 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:23.971496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68767026-cd42-4a38-a998-1ec295257cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:23.972124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68767026-cd42-4a38-a998-1ec295257cfb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:23.972403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68767026-cd42-4a38-a998-1ec295257cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:23.972669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68767026-cd42-4a38-a998-1ec295257cfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:23.979401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68767026-cd42-4a38-a998-1ec295257cfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:23.980863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68767026-cd42-4a38-a998-1ec295257cfb None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-68767026-cd42-4a38-a998-1ec295257cfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:46:23.994146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:24.000465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:24.001643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:46:24.001643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:46:24.001643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:46:24.001643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:46:24.001643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:46:24.001643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:46:24.001643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:24.001643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:24.001643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 616/1230] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:24.002769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942b0c9-5575-417b-925c-2373fd6aad2a None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:24.110047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942b0c9-5575-417b-925c-2373fd6aad2a None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:24.146729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6539e1f-b77b-4131-825e-4bf435c1a55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:24.147333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6539e1f-b77b-4131-825e-4bf435c1a55f None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:24.147609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6539e1f-b77b-4131-825e-4bf435c1a55f None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:24.147874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6539e1f-b77b-4131-825e-4bf435c1a55f None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:24.148555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6539e1f-b77b-4131-825e-4bf435c1a55f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:24.148916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6539e1f-b77b-4131-825e-4bf435c1a55f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:24.150361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6539e1f-b77b-4131-825e-4bf435c1a55f None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:24.164464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 615/1231] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:46:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:46:24.170586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68767026-cd42-4a38-a998-1ec295257cfb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:24.245474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:24.283049 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68767026-cd42-4a38-a998-1ec295257cfb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:24.319158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4384f0-7e8c-4ece-af5c-08f32de522cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:24.320444 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4384f0-7e8c-4ece-af5c-08f32de522cc None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:24.320955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4384f0-7e8c-4ece-af5c-08f32de522cc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:24.321378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4384f0-7e8c-4ece-af5c-08f32de522cc None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/0620b2af88034caa8de99e9b48b05f7e/roles/fa23cf1cb845457d9460c2a9884af4cf` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:24.338282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4384f0-7e8c-4ece-af5c-08f32de522cc None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=0620b2af88034caa8de99e9b48b05f7e, role_id=fa23cf1cb845457d9460c2a9884af4cf)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:24.339733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4384f0-7e8c-4ece-af5c-08f32de522cc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:24.352309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 617/1232] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:46:24 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/0620b2af88034caa8de99e9b48b05f7e/roles/fa23cf1cb845457d9460c2a9884af4cf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:46:26.652052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d46bdc-3955-4029-9be7-318f37a934e3 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:26.652586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d46bdc-3955-4029-9be7-318f37a934e3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:26.652586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d46bdc-3955-4029-9be7-318f37a934e3 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:26.654017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 616/1233] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:26.660087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87591e28-9b10-4ff3-af23-9d1249987fe6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:26.660549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87591e28-9b10-4ff3-af23-9d1249987fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:26.660687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87591e28-9b10-4ff3-af23-9d1249987fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:26.680911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-87591e28-9b10-4ff3-af23-9d1249987fe6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:26.682127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-87591e28-9b10-4ff3-af23-9d1249987fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:26.758790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 618/1234] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:27.062435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d46bdc-3955-4029-9be7-318f37a934e3 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:27.090204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:27.175302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d46bdc-3955-4029-9be7-318f37a934e3 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:27.215173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce18d5-e5c6-4168-a451-62e106577ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:27.215835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce18d5-e5c6-4168-a451-62e106577ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:27.216084 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce18d5-e5c6-4168-a451-62e106577ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:27.216339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce18d5-e5c6-4168-a451-62e106577ba2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:27.231311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce18d5-e5c6-4168-a451-62e106577ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-39ce18d5-e5c6-4168-a451-62e106577ba2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:46:27.237798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:27.238953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:27.239959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 617/1235] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Sep 15 10:46:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:27.245848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87591e28-9b10-4ff3-af23-9d1249987fe6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:27.299638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:27.357561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87591e28-9b10-4ff3-af23-9d1249987fe6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:27.393493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4363203-24ad-4d50-a55a-fae44dfe6303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:27.394101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4363203-24ad-4d50-a55a-fae44dfe6303 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:27.394376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4363203-24ad-4d50-a55a-fae44dfe6303 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:27.394641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4363203-24ad-4d50-a55a-fae44dfe6303 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:27.395314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4363203-24ad-4d50-a55a-fae44dfe6303 None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:27.395628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4363203-24ad-4d50-a55a-fae44dfe6303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:27.397224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4363203-24ad-4d50-a55a-fae44dfe6303 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:27.406145 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 619/1236] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Sep 15 10:46:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:27.412315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce18d5-e5c6-4168-a451-62e106577ba2 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:27.507005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:27.524824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce18d5-e5c6-4168-a451-62e106577ba2 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:27.562356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275b498-e52c-43c6-be5e-f7e3b7751d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:27.562973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275b498-e52c-43c6-be5e-f7e3b7751d85 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:27.563251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275b498-e52c-43c6-be5e-f7e3b7751d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:27.563589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275b498-e52c-43c6-be5e-f7e3b7751d85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:27.574859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275b498-e52c-43c6-be5e-f7e3b7751d85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:27.576313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275b498-e52c-43c6-be5e-f7e3b7751d85 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-e275b498-e52c-43c6-be5e-f7e3b7751d85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:27.592028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:27.593211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:27.594167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:27.594167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:27.594167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:27.594167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 618/1237] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:27.599941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4363203-24ad-4d50-a55a-fae44dfe6303 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:27.708183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4363203-24ad-4d50-a55a-fae44dfe6303 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:27.717114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:27.745303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d3ffe-8f5c-4a59-94e0-6db54f2d3cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:27.745999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d3ffe-8f5c-4a59-94e0-6db54f2d3cbe None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:27.746303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d3ffe-8f5c-4a59-94e0-6db54f2d3cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:27.746594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d3ffe-8f5c-4a59-94e0-6db54f2d3cbe None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:27.747310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d3ffe-8f5c-4a59-94e0-6db54f2d3cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:27.747626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d3ffe-8f5c-4a59-94e0-6db54f2d3cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:27.749118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d3ffe-8f5c-4a59-94e0-6db54f2d3cbe None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:27.762763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 620/1238] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:27.768739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275b498-e52c-43c6-be5e-f7e3b7751d85 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:27.878397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275b498-e52c-43c6-be5e-f7e3b7751d85 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:27.917732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c79e50-8745-495a-bcae-d75be8906ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:27.918596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c79e50-8745-495a-bcae-d75be8906ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:27.918911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c79e50-8745-495a-bcae-d75be8906ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:27.919203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c79e50-8745-495a-bcae-d75be8906ad8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:27.922833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:27.929276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c79e50-8745-495a-bcae-d75be8906ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:27.931126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c79e50-8745-495a-bcae-d75be8906ad8 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-55c79e50-8745-495a-bcae-d75be8906ad8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:46:27.943098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:27.944267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:27.945569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 619/1239] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:46:27.950675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d3ffe-8f5c-4a59-94e0-6db54f2d3cbe None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:28.059071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d3ffe-8f5c-4a59-94e0-6db54f2d3cbe None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:28.099845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8982c-34e9-4b7d-a445-9add9c1206e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:28.100517 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8982c-34e9-4b7d-a445-9add9c1206e1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:28.100871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8982c-34e9-4b7d-a445-9add9c1206e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:28.101193 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8982c-34e9-4b7d-a445-9add9c1206e1 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:28.101937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8982c-34e9-4b7d-a445-9add9c1206e1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:28.102244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8982c-34e9-4b7d-a445-9add9c1206e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:28.103837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8982c-34e9-4b7d-a445-9add9c1206e1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:28.121360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 621/1240] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Sep 15 10:46:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:28.128929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c79e50-8745-495a-bcae-d75be8906ad8 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:28.239735 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c79e50-8745-495a-bcae-d75be8906ad8 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:28.279940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3764430-f265-401e-8bd3-5ef599e78378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:28.280613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3764430-f265-401e-8bd3-5ef599e78378 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:28.281034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3764430-f265-401e-8bd3-5ef599e78378 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:28.281288 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3764430-f265-401e-8bd3-5ef599e78378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:28.352332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:28.361747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3764430-f265-401e-8bd3-5ef599e78378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:28.363377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3764430-f265-401e-8bd3-5ef599e78378 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:28.471192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 620/1241] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:46:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:46:28.477505 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8982c-34e9-4b7d-a445-9add9c1206e1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:28.507998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:28.589520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8982c-34e9-4b7d-a445-9add9c1206e1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:28.629455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7d7f-6b94-49bd-9bed-4448d9839486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:28.630006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7d7f-6b94-49bd-9bed-4448d9839486 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:28.630284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7d7f-6b94-49bd-9bed-4448d9839486 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:28.630643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7d7f-6b94-49bd-9bed-4448d9839486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:28.641791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7d7f-6b94-49bd-9bed-4448d9839486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:28.643709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7d7f-6b94-49bd-9bed-4448d9839486 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5eba7d7f-6b94-49bd-9bed-4448d9839486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:28.659610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:28.660713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:28.661748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:28.661748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:28.661748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:28.661748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 622/1242] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:28.666481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3764430-f265-401e-8bd3-5ef599e78378 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:28.777820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:28.784926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3764430-f265-401e-8bd3-5ef599e78378 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:28.823044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba602a07-330b-4bd4-8db8-23bf71a38fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:28.823790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba602a07-330b-4bd4-8db8-23bf71a38fef None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:28.824075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba602a07-330b-4bd4-8db8-23bf71a38fef None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:28.824352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba602a07-330b-4bd4-8db8-23bf71a38fef None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:28.825190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba602a07-330b-4bd4-8db8-23bf71a38fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:28.825541 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba602a07-330b-4bd4-8db8-23bf71a38fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:28.826984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba602a07-330b-4bd4-8db8-23bf71a38fef None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:28.839105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 621/1243] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:28.845533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7d7f-6b94-49bd-9bed-4448d9839486 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:28.933740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:28.959653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7d7f-6b94-49bd-9bed-4448d9839486 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:28.997536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2780b-0e06-4b14-bd5b-dd1dd3d9eee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:28.998402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2780b-0e06-4b14-bd5b-dd1dd3d9eee5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:28.998781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2780b-0e06-4b14-bd5b-dd1dd3d9eee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:28.999180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2780b-0e06-4b14-bd5b-dd1dd3d9eee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:29.009552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2780b-0e06-4b14-bd5b-dd1dd3d9eee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:29.011388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2780b-0e06-4b14-bd5b-dd1dd3d9eee5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-27a2780b-0e06-4b14-bd5b-dd1dd3d9eee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:46:29.023163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:29.024290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:29.025589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:46:29.025589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:46:29.025589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:46:29.025589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:46:29.025589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:46:29.025589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:46:29.025589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:29.025589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:29.025589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 623/1244] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:46:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:29.030696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba602a07-330b-4bd4-8db8-23bf71a38fef None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:29.145833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba602a07-330b-4bd4-8db8-23bf71a38fef None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:29.184136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3af73-688d-4b66-b5ff-c464b7ed3566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:29.184760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3af73-688d-4b66-b5ff-c464b7ed3566 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:29.185071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3af73-688d-4b66-b5ff-c464b7ed3566 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:29.185377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3af73-688d-4b66-b5ff-c464b7ed3566 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:29.186127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3af73-688d-4b66-b5ff-c464b7ed3566 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:29.186449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3af73-688d-4b66-b5ff-c464b7ed3566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:29.187982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3af73-688d-4b66-b5ff-c464b7ed3566 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:29.202532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 622/1245] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:29.209684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2780b-0e06-4b14-bd5b-dd1dd3d9eee5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:29.330231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2780b-0e06-4b14-bd5b-dd1dd3d9eee5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:29.366565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc7355-53a2-40c6-885c-f75aca0af502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:29.367177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc7355-53a2-40c6-885c-f75aca0af502 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:29.367463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc7355-53a2-40c6-885c-f75aca0af502 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:29.367734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc7355-53a2-40c6-885c-f75aca0af502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:29.368454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc7355-53a2-40c6-885c-f75aca0af502 None admin] RBAC: Adding query filter params (role.id=fa23cf1cb845457d9460c2a9884af4cf, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=0620b2af88034caa8de99e9b48b05f7e) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:29.368761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc7355-53a2-40c6-885c-f75aca0af502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:29.370233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc7355-53a2-40c6-885c-f75aca0af502 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:29.377088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 624/1246] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:46:29 2022] GET /identity/v3/role_assignments?user.id=0620b2af88034caa8de99e9b48b05f7e&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=fa23cf1cb845457d9460c2a9884af4cf => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:31.819279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c5e1f-f525-4233-9b2e-b16069b901a1 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:31.820053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c5e1f-f525-4233-9b2e-b16069b901a1 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:31.820053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c5e1f-f525-4233-9b2e-b16069b901a1 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:31.821536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 623/1247] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:31.826512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f29d7-54f8-4364-a494-1c7792f52205 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:31.826911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f29d7-54f8-4364-a494-1c7792f52205 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:31.827054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f29d7-54f8-4364-a494-1c7792f52205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:31.834393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:31.847906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c53f29d7-54f8-4364-a494-1c7792f52205 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:31.849149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c53f29d7-54f8-4364-a494-1c7792f52205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:31.925320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 625/1248] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:32.204348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c5e1f-f525-4233-9b2e-b16069b901a1 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:32.224662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:32.321161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c5e1f-f525-4233-9b2e-b16069b901a1 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:32.360103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e63fd5-389e-4ba0-ba63-eaa226c0bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:32.361524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e63fd5-389e-4ba0-ba63-eaa226c0bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:32.361958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e63fd5-389e-4ba0-ba63-eaa226c0bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:32.362219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e63fd5-389e-4ba0-ba63-eaa226c0bdf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:32.373273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e63fd5-389e-4ba0-ba63-eaa226c0bdf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-a4e63fd5-389e-4ba0-ba63-eaa226c0bdf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:46:32.379116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:32.380172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:32.381181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 624/1249] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:46:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 10:46:32.386470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f29d7-54f8-4364-a494-1c7792f52205 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:32.497094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f29d7-54f8-4364-a494-1c7792f52205 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:32.528586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:32.535991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80014e1-fdcc-4504-80b1-3969db9d43c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:32.536713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80014e1-fdcc-4504-80b1-3969db9d43c7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:32.537170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80014e1-fdcc-4504-80b1-3969db9d43c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:32.537504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80014e1-fdcc-4504-80b1-3969db9d43c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:32.538480 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80014e1-fdcc-4504-80b1-3969db9d43c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:32.538812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80014e1-fdcc-4504-80b1-3969db9d43c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:32.540629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80014e1-fdcc-4504-80b1-3969db9d43c7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:32.550783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 626/1250] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:46:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:32.557026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e63fd5-389e-4ba0-ba63-eaa226c0bdf8 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:32.647823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:32.673728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e63fd5-389e-4ba0-ba63-eaa226c0bdf8 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:32.709994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938bf58-8a72-49bc-ad39-c1afeed83489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:32.710629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938bf58-8a72-49bc-ad39-c1afeed83489 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:32.710907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938bf58-8a72-49bc-ad39-c1afeed83489 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:32.711177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938bf58-8a72-49bc-ad39-c1afeed83489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:32.722241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938bf58-8a72-49bc-ad39-c1afeed83489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:32.723700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938bf58-8a72-49bc-ad39-c1afeed83489 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-a938bf58-8a72-49bc-ad39-c1afeed83489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:32.740050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:32.741333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:32.742244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:32.742244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:32.742244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:32.742244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 625/1251] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:32.747539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80014e1-fdcc-4504-80b1-3969db9d43c7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:32.853475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80014e1-fdcc-4504-80b1-3969db9d43c7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:32.889995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdc1f7-4f76-42f8-8c6d-fcdd72f3d23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:32.890632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdc1f7-4f76-42f8-8c6d-fcdd72f3d23d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:32.890929 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdc1f7-4f76-42f8-8c6d-fcdd72f3d23d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:32.891189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdc1f7-4f76-42f8-8c6d-fcdd72f3d23d None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:32.892067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdc1f7-4f76-42f8-8c6d-fcdd72f3d23d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:32.892394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdc1f7-4f76-42f8-8c6d-fcdd72f3d23d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:32.893859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdc1f7-4f76-42f8-8c6d-fcdd72f3d23d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:32.907371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 627/1252] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:32.913366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938bf58-8a72-49bc-ad39-c1afeed83489 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:33.025294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938bf58-8a72-49bc-ad39-c1afeed83489 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:33.051777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:33.059892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af350b8-47a5-4071-8628-f01019258dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:33.060518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af350b8-47a5-4071-8628-f01019258dba None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:33.060816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af350b8-47a5-4071-8628-f01019258dba None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:33.061094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af350b8-47a5-4071-8628-f01019258dba None admin] PATH_INFO: `/v3/users/placement` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:33.070027 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af350b8-47a5-4071-8628-f01019258dba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:33.071473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af350b8-47a5-4071-8628-f01019258dba None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-1af350b8-47a5-4071-8628-f01019258dba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:46:33.083506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:33.084668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:33.085967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:33.088885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 626/1253] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:33.090812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdc1f7-4f76-42f8-8c6d-fcdd72f3d23d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:33.148883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:33.193892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdc1f7-4f76-42f8-8c6d-fcdd72f3d23d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:33.236586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b7df3-f848-4b97-be85-1f01c28e3f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:33.237243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b7df3-f848-4b97-be85-1f01c28e3f02 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:33.237529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b7df3-f848-4b97-be85-1f01c28e3f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:33.237815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b7df3-f848-4b97-be85-1f01c28e3f02 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:33.238545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b7df3-f848-4b97-be85-1f01c28e3f02 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:33.238852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b7df3-f848-4b97-be85-1f01c28e3f02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:33.240361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b7df3-f848-4b97-be85-1f01c28e3f02 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:33.260939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 628/1254] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Sep 15 10:46:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:33.268905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af350b8-47a5-4071-8628-f01019258dba None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:33.370620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af350b8-47a5-4071-8628-f01019258dba None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:33.403867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137170d-d5f5-45b7-bc48-fb98c747185e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:33.404467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137170d-d5f5-45b7-bc48-fb98c747185e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:33.404752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137170d-d5f5-45b7-bc48-fb98c747185e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:33.405056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137170d-d5f5-45b7-bc48-fb98c747185e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:33.458086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:33.470474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137170d-d5f5-45b7-bc48-fb98c747185e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:33.471916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137170d-d5f5-45b7-bc48-fb98c747185e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:33.566763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 627/1255] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:46:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:46:33.572822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b7df3-f848-4b97-be85-1f01c28e3f02 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:33.670021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b7df3-f848-4b97-be85-1f01c28e3f02 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:33.683226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:33.705355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81108b7d-ed48-4af7-86ec-ddf2f8b312db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:33.705977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108b7d-ed48-4af7-86ec-ddf2f8b312db None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:33.706259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108b7d-ed48-4af7-86ec-ddf2f8b312db None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:33.706531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108b7d-ed48-4af7-86ec-ddf2f8b312db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:33.716176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81108b7d-ed48-4af7-86ec-ddf2f8b312db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:33.717643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81108b7d-ed48-4af7-86ec-ddf2f8b312db None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-81108b7d-ed48-4af7-86ec-ddf2f8b312db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:33.731939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:33.733088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:33.734187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:33.734187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:33.734187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:33.734187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 629/1256] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:33.739273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137170d-d5f5-45b7-bc48-fb98c747185e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:33.824839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:33.836907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137170d-d5f5-45b7-bc48-fb98c747185e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:33.869866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2b743-54aa-4a59-b432-5bdaa68a0454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:33.870465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2b743-54aa-4a59-b432-5bdaa68a0454 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:33.870747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2b743-54aa-4a59-b432-5bdaa68a0454 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:33.871021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2b743-54aa-4a59-b432-5bdaa68a0454 None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:33.871664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2b743-54aa-4a59-b432-5bdaa68a0454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:33.871970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2b743-54aa-4a59-b432-5bdaa68a0454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:33.873377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2b743-54aa-4a59-b432-5bdaa68a0454 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:33.885150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 628/1257] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:33.890912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81108b7d-ed48-4af7-86ec-ddf2f8b312db None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:33.988947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81108b7d-ed48-4af7-86ec-ddf2f8b312db None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:34.020845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9aafcf-ae55-4fdc-8aa1-1146f2a986ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:34.021459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9aafcf-ae55-4fdc-8aa1-1146f2a986ce None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:34.021736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9aafcf-ae55-4fdc-8aa1-1146f2a986ce None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:34.022006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9aafcf-ae55-4fdc-8aa1-1146f2a986ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:34.028715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9aafcf-ae55-4fdc-8aa1-1146f2a986ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:34.030179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9aafcf-ae55-4fdc-8aa1-1146f2a986ce None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-7d9aafcf-ae55-4fdc-8aa1-1146f2a986ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:46:34.041766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:34.042819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:34.043949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:46:34.043949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:46:34.043949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:46:34.043949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:46:34.043949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:46:34.043949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:46:34.043949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:34.043949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:34.043949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 630/1258] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:34.050636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2b743-54aa-4a59-b432-5bdaa68a0454 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:34.147877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2b743-54aa-4a59-b432-5bdaa68a0454 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:34.181734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67f48a-7d9c-4511-a919-1a6d11f68740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:34.182328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67f48a-7d9c-4511-a919-1a6d11f68740 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:34.182607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67f48a-7d9c-4511-a919-1a6d11f68740 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:34.182877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67f48a-7d9c-4511-a919-1a6d11f68740 None admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:34.183557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67f48a-7d9c-4511-a919-1a6d11f68740 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:34.183859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67f48a-7d9c-4511-a919-1a6d11f68740 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:34.185313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67f48a-7d9c-4511-a919-1a6d11f68740 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:34.197172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 629/1259] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:46:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:34.203286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9aafcf-ae55-4fdc-8aa1-1146f2a986ce None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:34.237991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:34.318369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9aafcf-ae55-4fdc-8aa1-1146f2a986ce None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:34.352666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937c887-600c-4314-865e-7c20619a9bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:34.353665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937c887-600c-4314-865e-7c20619a9bdd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:34.354116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937c887-600c-4314-865e-7c20619a9bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:34.354567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937c887-600c-4314-865e-7c20619a9bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:34.355292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937c887-600c-4314-865e-7c20619a9bdd None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=0620b2af88034caa8de99e9b48b05f7e) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:34.355601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937c887-600c-4314-865e-7c20619a9bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:34.357849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937c887-600c-4314-865e-7c20619a9bdd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:34.364198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 631/1260] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:46:34 2022] GET /identity/v3/role_assignments?user.id=0620b2af88034caa8de99e9b48b05f7e&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=0be9777002c641b4a47f78df647cc173 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:36.614134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06487e2d-6fd5-427d-9ef5-32d5a4e20cbd None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:36.614643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06487e2d-6fd5-427d-9ef5-32d5a4e20cbd None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:36.614747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06487e2d-6fd5-427d-9ef5-32d5a4e20cbd None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:36.616052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 630/1261] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:36.621104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b8446-23b6-4611-ae5c-01ea9b84c973 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:36.621515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b8446-23b6-4611-ae5c-01ea9b84c973 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:36.621669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b8446-23b6-4611-ae5c-01ea9b84c973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:36.640322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9a2b8446-23b6-4611-ae5c-01ea9b84c973 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:36.642096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9a2b8446-23b6-4611-ae5c-01ea9b84c973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:36.707065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 632/1262] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:36.974607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06487e2d-6fd5-427d-9ef5-32d5a4e20cbd None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:36.985489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:37.075423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06487e2d-6fd5-427d-9ef5-32d5a4e20cbd None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:37.108069 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c0c10-ff3d-4d4d-b20e-ac763a2de7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:37.108682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c0c10-ff3d-4d4d-b20e-ac763a2de7da None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:37.109075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c0c10-ff3d-4d4d-b20e-ac763a2de7da None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:37.109352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c0c10-ff3d-4d4d-b20e-ac763a2de7da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:37.119226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c0c10-ff3d-4d4d-b20e-ac763a2de7da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-f33c0c10-ff3d-4d4d-b20e-ac763a2de7da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:46:37.124029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:37.125183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:37.126826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 631/1263] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:46:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:37.131883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b8446-23b6-4611-ae5c-01ea9b84c973 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:37.150309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:37.232981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b8446-23b6-4611-ae5c-01ea9b84c973 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:37.275431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a4bd3-abb0-4d6a-9378-871f3677ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:37.276170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a4bd3-abb0-4d6a-9378-871f3677ef92 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:37.276483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a4bd3-abb0-4d6a-9378-871f3677ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:37.276762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a4bd3-abb0-4d6a-9378-871f3677ef92 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:37.277630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a4bd3-abb0-4d6a-9378-871f3677ef92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:37.277974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a4bd3-abb0-4d6a-9378-871f3677ef92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:37.279632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a4bd3-abb0-4d6a-9378-871f3677ef92 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:37.289062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 633/1264] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:37.295781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c0c10-ff3d-4d4d-b20e-ac763a2de7da None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:37.362616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:37.396697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c0c10-ff3d-4d4d-b20e-ac763a2de7da None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:37.435042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd066d-87ee-4c3b-b169-66e27fadba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:37.435660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd066d-87ee-4c3b-b169-66e27fadba1b None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:37.435939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd066d-87ee-4c3b-b169-66e27fadba1b None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:37.436210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd066d-87ee-4c3b-b169-66e27fadba1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:37.447346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd066d-87ee-4c3b-b169-66e27fadba1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:37.448823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd066d-87ee-4c3b-b169-66e27fadba1b None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-afcd066d-87ee-4c3b-b169-66e27fadba1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:37.462571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:37.463652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:37.464617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:37.464617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:37.464617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:37.464617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 632/1265] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:37.469569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a4bd3-abb0-4d6a-9378-871f3677ef92 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:37.550994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:37.571455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a4bd3-abb0-4d6a-9378-871f3677ef92 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:37.603925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ceba4-8308-4dc2-958a-9802a2627a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:37.604532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ceba4-8308-4dc2-958a-9802a2627a01 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:37.604835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ceba4-8308-4dc2-958a-9802a2627a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:37.605114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ceba4-8308-4dc2-958a-9802a2627a01 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:37.605746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ceba4-8308-4dc2-958a-9802a2627a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:37.606051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ceba4-8308-4dc2-958a-9802a2627a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:37.607424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ceba4-8308-4dc2-958a-9802a2627a01 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:37.619239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 634/1266] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:37.624915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd066d-87ee-4c3b-b169-66e27fadba1b None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:37.724638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd066d-87ee-4c3b-b169-66e27fadba1b None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:37.745567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:37.760419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc81a7-4151-4813-8163-02fd9280f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:37.761069 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc81a7-4151-4813-8163-02fd9280f363 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:37.761349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc81a7-4151-4813-8163-02fd9280f363 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:37.761620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc81a7-4151-4813-8163-02fd9280f363 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:37.768854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc81a7-4151-4813-8163-02fd9280f363 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:37.770296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc81a7-4151-4813-8163-02fd9280f363 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-6dbc81a7-4151-4813-8163-02fd9280f363 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:46:37.781159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:37.782872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:37.784243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 633/1267] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:37.788667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ceba4-8308-4dc2-958a-9802a2627a01 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:37.885734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ceba4-8308-4dc2-958a-9802a2627a01 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:37.918994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14625c9-98d0-4d30-b389-1a712005a340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:37.919590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14625c9-98d0-4d30-b389-1a712005a340 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:37.919870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14625c9-98d0-4d30-b389-1a712005a340 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:37.920142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14625c9-98d0-4d30-b389-1a712005a340 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:37.920879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14625c9-98d0-4d30-b389-1a712005a340 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:37.921184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14625c9-98d0-4d30-b389-1a712005a340 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:37.923223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14625c9-98d0-4d30-b389-1a712005a340 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:37.931655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:37.940256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 635/1268] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Sep 15 10:46:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:37.946359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc81a7-4151-4813-8163-02fd9280f363 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:38.045022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc81a7-4151-4813-8163-02fd9280f363 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:38.078071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49698b6f-3342-4dbf-b2a3-76e4866cfc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:38.078716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49698b6f-3342-4dbf-b2a3-76e4866cfc8e None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:38.079095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49698b6f-3342-4dbf-b2a3-76e4866cfc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:38.079371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49698b6f-3342-4dbf-b2a3-76e4866cfc8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:38.089488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49698b6f-3342-4dbf-b2a3-76e4866cfc8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:38.090924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49698b6f-3342-4dbf-b2a3-76e4866cfc8e None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-49698b6f-3342-4dbf-b2a3-76e4866cfc8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:38.105067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:38.106211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:38.107380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:38.107380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:38.107380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:38.107380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 634/1269] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:38.112611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14625c9-98d0-4d30-b389-1a712005a340 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:38.217081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14625c9-98d0-4d30-b389-1a712005a340 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:38.253125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc75c5-d67c-49db-b902-44e0724d0258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:38.253732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc75c5-d67c-49db-b902-44e0724d0258 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:38.254013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc75c5-d67c-49db-b902-44e0724d0258 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:38.254285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc75c5-d67c-49db-b902-44e0724d0258 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:38.254915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc75c5-d67c-49db-b902-44e0724d0258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:38.255218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc75c5-d67c-49db-b902-44e0724d0258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:38.256597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc75c5-d67c-49db-b902-44e0724d0258 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:38.268203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 636/1270] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:38.273969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49698b6f-3342-4dbf-b2a3-76e4866cfc8e None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:38.305482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:38.373453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49698b6f-3342-4dbf-b2a3-76e4866cfc8e None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:38.405889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340f927-2302-4144-98a4-0bb5d8017d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:38.406509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340f927-2302-4144-98a4-0bb5d8017d87 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:38.407438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340f927-2302-4144-98a4-0bb5d8017d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:38.407716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340f927-2302-4144-98a4-0bb5d8017d87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:38.414181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340f927-2302-4144-98a4-0bb5d8017d87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:38.415629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340f927-2302-4144-98a4-0bb5d8017d87 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-2340f927-2302-4144-98a4-0bb5d8017d87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:46:38.426855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:38.427925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:38.429072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:46:38.429072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:46:38.429072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:46:38.429072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:46:38.429072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:46:38.429072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:46:38.429072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:38.429072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:38.429072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 635/1271] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:46:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:38.433726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc75c5-d67c-49db-b902-44e0724d0258 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:38.492100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:38.534029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc75c5-d67c-49db-b902-44e0724d0258 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:38.567287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e86521-366a-46e1-a815-a9797f3eaaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:38.567892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e86521-366a-46e1-a815-a9797f3eaaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:38.568173 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e86521-366a-46e1-a815-a9797f3eaaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:38.568445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e86521-366a-46e1-a815-a9797f3eaaa1 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:38.569415 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e86521-366a-46e1-a815-a9797f3eaaa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:38.569725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e86521-366a-46e1-a815-a9797f3eaaa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:38.571155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e86521-366a-46e1-a815-a9797f3eaaa1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:38.583435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 637/1272] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:46:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:38.589392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340f927-2302-4144-98a4-0bb5d8017d87 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:38.679442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:38.689047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340f927-2302-4144-98a4-0bb5d8017d87 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:38.721902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7de30-dbc5-4da5-b45b-7cd37e0cc6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:38.722536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7de30-dbc5-4da5-b45b-7cd37e0cc6e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:38.722812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7de30-dbc5-4da5-b45b-7cd37e0cc6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:38.723082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7de30-dbc5-4da5-b45b-7cd37e0cc6e4 None admin] PATH_INFO: `/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/0620b2af88034caa8de99e9b48b05f7e/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:38.724945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7de30-dbc5-4da5-b45b-7cd37e0cc6e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f1eafaf73ac46c68507d601a32598f5, user_id=0620b2af88034caa8de99e9b48b05f7e, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:38.726364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7de30-dbc5-4da5-b45b-7cd37e0cc6e4 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:38.738503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 636/1273] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Sep 15 10:46:38 2022] PUT /identity/v3/projects/1f1eafaf73ac46c68507d601a32598f5/users/0620b2af88034caa8de99e9b48b05f7e/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:46:40.864457 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97786609-eb59-4bd7-99f0-b3baaf91c046 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:40.865004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97786609-eb59-4bd7-99f0-b3baaf91c046 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:40.865004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97786609-eb59-4bd7-99f0-b3baaf91c046 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:40.866367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 638/1274] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:40.871610 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c00134-abe4-482a-b10e-e6a9f1b37859 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:40.872014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c00134-abe4-482a-b10e-e6a9f1b37859 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:40.872194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c00134-abe4-482a-b10e-e6a9f1b37859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:40.890706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-65c00134-abe4-482a-b10e-e6a9f1b37859 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:40.892554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-65c00134-abe4-482a-b10e-e6a9f1b37859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:40.958071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 637/1275] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:41.228738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97786609-eb59-4bd7-99f0-b3baaf91c046 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:41.333355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97786609-eb59-4bd7-99f0-b3baaf91c046 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:41.350151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:41.367030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905bfa4-218b-46e0-bfe9-baae03670961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:41.367654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905bfa4-218b-46e0-bfe9-baae03670961 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:41.367935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905bfa4-218b-46e0-bfe9-baae03670961 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:41.368207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905bfa4-218b-46e0-bfe9-baae03670961 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:41.378636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905bfa4-218b-46e0-bfe9-baae03670961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6905bfa4-218b-46e0-bfe9-baae03670961 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 10:46:41.383420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:41.384506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:41.385746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 639/1276] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Sep 15 10:46:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:41.390559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c00134-abe4-482a-b10e-e6a9f1b37859 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:41.468073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:41.487671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c00134-abe4-482a-b10e-e6a9f1b37859 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:41.520383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fca5e8-54da-4057-b3b2-0c4e9dfd86a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:41.521008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fca5e8-54da-4057-b3b2-0c4e9dfd86a2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:41.521291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fca5e8-54da-4057-b3b2-0c4e9dfd86a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:41.521564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fca5e8-54da-4057-b3b2-0c4e9dfd86a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:41.522222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fca5e8-54da-4057-b3b2-0c4e9dfd86a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:41.522527 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fca5e8-54da-4057-b3b2-0c4e9dfd86a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:41.523882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fca5e8-54da-4057-b3b2-0c4e9dfd86a2 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:41.532217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 638/1277] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Sep 15 10:46:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:41.538223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905bfa4-218b-46e0-bfe9-baae03670961 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:41.636574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905bfa4-218b-46e0-bfe9-baae03670961 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:41.669801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c3803-38a2-40e7-aeb4-e638ee52d94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:41.670422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c3803-38a2-40e7-aeb4-e638ee52d94c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:41.670703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c3803-38a2-40e7-aeb4-e638ee52d94c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:41.670975 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c3803-38a2-40e7-aeb4-e638ee52d94c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:41.681399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c3803-38a2-40e7-aeb4-e638ee52d94c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:41.682844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c3803-38a2-40e7-aeb4-e638ee52d94c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-333c3803-38a2-40e7-aeb4-e638ee52d94c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:41.697385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:41.698584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:41.699538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:41.699538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:41.699538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:41.699538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 640/1278] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:41.704330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fca5e8-54da-4057-b3b2-0c4e9dfd86a2 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:41.801669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fca5e8-54da-4057-b3b2-0c4e9dfd86a2 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:41.834012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbcd4a-2e46-4bc0-bb77-c17c4d8a4c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:41.834602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbcd4a-2e46-4bc0-bb77-c17c4d8a4c3c None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:41.834881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbcd4a-2e46-4bc0-bb77-c17c4d8a4c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:41.835153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbcd4a-2e46-4bc0-bb77-c17c4d8a4c3c None admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:41.835782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbbcd4a-2e46-4bc0-bb77-c17c4d8a4c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:41.836081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbbcd4a-2e46-4bc0-bb77-c17c4d8a4c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:41.837476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbbcd4a-2e46-4bc0-bb77-c17c4d8a4c3c None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:41.849218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 639/1279] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:41.854799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c3803-38a2-40e7-aeb4-e638ee52d94c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:41.885707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:41.954994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c3803-38a2-40e7-aeb4-e638ee52d94c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:41.987733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fbc7d-5f74-4377-8a87-5d70005567ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:41.988344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fbc7d-5f74-4377-8a87-5d70005567ba None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:41.988626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fbc7d-5f74-4377-8a87-5d70005567ba None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:41.988927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fbc7d-5f74-4377-8a87-5d70005567ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:41.996001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fbc7d-5f74-4377-8a87-5d70005567ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:41.997493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fbc7d-5f74-4377-8a87-5d70005567ba None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3d6fbc7d-5f74-4377-8a87-5d70005567ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 10:46:42.008384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:42.009543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:42.010546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 641/1280] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:42.015366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbcd4a-2e46-4bc0-bb77-c17c4d8a4c3c None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:42.019780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:42.113760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbcd4a-2e46-4bc0-bb77-c17c4d8a4c3c None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:42.146360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2ed50-b11d-4622-b7e1-0a90a601edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:42.146951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2ed50-b11d-4622-b7e1-0a90a601edea None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:42.147227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2ed50-b11d-4622-b7e1-0a90a601edea None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:42.147496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2ed50-b11d-4622-b7e1-0a90a601edea None admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:42.148207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2ed50-b11d-4622-b7e1-0a90a601edea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:42.148509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2ed50-b11d-4622-b7e1-0a90a601edea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:42.149956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2ed50-b11d-4622-b7e1-0a90a601edea None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:42.166917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 640/1281] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Sep 15 10:46:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:42.172990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fbc7d-5f74-4377-8a87-5d70005567ba None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:42.264292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:42.274680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fbc7d-5f74-4377-8a87-5d70005567ba None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:42.306608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc479bd-4c6e-48ac-b8f1-a14fed145903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:42.307208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc479bd-4c6e-48ac-b8f1-a14fed145903 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:42.307486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc479bd-4c6e-48ac-b8f1-a14fed145903 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:42.307755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc479bd-4c6e-48ac-b8f1-a14fed145903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:42.372473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc479bd-4c6e-48ac-b8f1-a14fed145903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:42.373908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc479bd-4c6e-48ac-b8f1-a14fed145903 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:42.459567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:42.470121 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 642/1282] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Sep 15 10:46:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 10:46:42.475884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2ed50-b11d-4622-b7e1-0a90a601edea None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:42.536700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:42.575259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2ed50-b11d-4622-b7e1-0a90a601edea None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:42.608294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946b117-d4de-47cc-a03a-c8004d4475a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:42.608952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946b117-d4de-47cc-a03a-c8004d4475a2 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:42.609231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946b117-d4de-47cc-a03a-c8004d4475a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:42.610029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946b117-d4de-47cc-a03a-c8004d4475a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:42.620694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946b117-d4de-47cc-a03a-c8004d4475a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:42.622174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946b117-d4de-47cc-a03a-c8004d4475a2 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-7946b117-d4de-47cc-a03a-c8004d4475a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 10:46:42.637082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:42.638244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 10:46:42.639381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 10:46:42.639381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 10:46:42.639381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:42.639381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 641/1283] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:42.644191 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc479bd-4c6e-48ac-b8f1-a14fed145903 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:42.742376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc479bd-4c6e-48ac-b8f1-a14fed145903 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:42.775941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c5d8b-fe9c-4bde-8d79-7d9dd8380611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:42.776540 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c5d8b-fe9c-4bde-8d79-7d9dd8380611 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:42.776843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c5d8b-fe9c-4bde-8d79-7d9dd8380611 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:42.777127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c5d8b-fe9c-4bde-8d79-7d9dd8380611 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:42.777757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c5d8b-fe9c-4bde-8d79-7d9dd8380611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:42.778059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c5d8b-fe9c-4bde-8d79-7d9dd8380611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:42.779432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c5d8b-fe9c-4bde-8d79-7d9dd8380611 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:42.790428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 643/1284] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Sep 15 10:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:42.796213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946b117-d4de-47cc-a03a-c8004d4475a2 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:42.894215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946b117-d4de-47cc-a03a-c8004d4475a2 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:42.907020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:42.927007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bba1a-d425-40e4-9cab-ae697756b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:42.927761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bba1a-d425-40e4-9cab-ae697756b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:42.927865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bba1a-d425-40e4-9cab-ae697756b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:42.928137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bba1a-d425-40e4-9cab-ae697756b4d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:42.934543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bba1a-d425-40e4-9cab-ae697756b4d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:42.935978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bba1a-d425-40e4-9cab-ae697756b4d9 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-ed8bba1a-d425-40e4-9cab-ae697756b4d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 10:46:42.947843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:42.948944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:42.950182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 10:46:42.950182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 10:46:42.950182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 10:46:42.950182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 10:46:42.950182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 10:46:42.950182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 10:46:42.950182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 10:46:42.950182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 10:46:42.950182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 642/1285] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Sep 15 10:46:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:42.955046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c5d8b-fe9c-4bde-8d79-7d9dd8380611 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:43.004528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:43.054774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c5d8b-fe9c-4bde-8d79-7d9dd8380611 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:43.087351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cdeb4-b565-43c9-9edf-30349fae1ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:43.087953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cdeb4-b565-43c9-9edf-30349fae1ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:43.088238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cdeb4-b565-43c9-9edf-30349fae1ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:43.088513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cdeb4-b565-43c9-9edf-30349fae1ba3 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:43.089225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cdeb4-b565-43c9-9edf-30349fae1ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:43.089531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cdeb4-b565-43c9-9edf-30349fae1ba3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:43.090965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cdeb4-b565-43c9-9edf-30349fae1ba3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:43.103217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 644/1286] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Sep 15 10:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:43.109347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bba1a-d425-40e4-9cab-ae697756b4d9 None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:43.205004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bba1a-d425-40e4-9cab-ae697756b4d9 None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:43.237977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6c20f-2022-46ac-9f95-e36de629a925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:43.238562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6c20f-2022-46ac-9f95-e36de629a925 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:43.238838 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6c20f-2022-46ac-9f95-e36de629a925 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:43.239107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6c20f-2022-46ac-9f95-e36de629a925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:43.239811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6c20f-2022-46ac-9f95-e36de629a925 None admin] RBAC: Adding query filter params (role.id=0be9777002c641b4a47f78df647cc173, scope.project.id=1f1eafaf73ac46c68507d601a32598f5, user.id=0620b2af88034caa8de99e9b48b05f7e) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:43.240114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6c20f-2022-46ac-9f95-e36de629a925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:43.241553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6c20f-2022-46ac-9f95-e36de629a925 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:43.247081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 643/1287] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Sep 15 10:46:43 2022] GET /identity/v3/role_assignments?user.id=0620b2af88034caa8de99e9b48b05f7e&scope.project.id=1f1eafaf73ac46c68507d601a32598f5&role.id=0be9777002c641b4a47f78df647cc173 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:45.401088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6832e11-6442-409a-833a-225a1a1fa44d None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:45.401622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6832e11-6442-409a-833a-225a1a1fa44d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:45.401622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6832e11-6442-409a-833a-225a1a1fa44d None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:45.402972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 645/1288] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:45.408142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4a753-8d00-47a7-a79f-01c3d185859c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:45.408582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4a753-8d00-47a7-a79f-01c3d185859c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:45.408677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4a753-8d00-47a7-a79f-01c3d185859c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:45.426523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-57d4a753-8d00-47a7-a79f-01c3d185859c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:45.427704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-57d4a753-8d00-47a7-a79f-01c3d185859c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:45.457512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:45.493385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 644/1289] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:45.761788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6832e11-6442-409a-833a-225a1a1fa44d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:45.863167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6832e11-6442-409a-833a-225a1a1fa44d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:45.873338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:45.902462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c288e-ef1c-445b-a3a0-a253c3433f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:45.903230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c288e-ef1c-445b-a3a0-a253c3433f34 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:45.903513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c288e-ef1c-445b-a3a0-a253c3433f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:45.903785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c288e-ef1c-445b-a3a0-a253c3433f34 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:45.904356 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c288e-ef1c-445b-a3a0-a253c3433f34 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:45.905779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c288e-ef1c-445b-a3a0-a253c3433f34 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-555c288e-ef1c-445b-a3a0-a253c3433f34 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:45.914883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:45.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:45.917266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 646/1290] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:46:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:45.921911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4a753-8d00-47a7-a79f-01c3d185859c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:46.018759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4a753-8d00-47a7-a79f-01c3d185859c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:46.052559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b3766-6f97-4756-bc51-2ccd1d8f56be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:46.053190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b3766-6f97-4756-bc51-2ccd1d8f56be None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:46.053467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b3766-6f97-4756-bc51-2ccd1d8f56be None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:46.053739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b3766-6f97-4756-bc51-2ccd1d8f56be None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:46.054351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b3766-6f97-4756-bc51-2ccd1d8f56be None admin] RBAC: Adding query filter params (name=placement) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:46.054650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b3766-6f97-4756-bc51-2ccd1d8f56be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:46.055998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b3766-6f97-4756-bc51-2ccd1d8f56be None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:46.060629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 645/1291] 10.222.0.50 () {56 vars in 1116 bytes} [Thu Sep 15 10:46:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:46.068131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c288e-ef1c-445b-a3a0-a253c3433f34 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:46.167200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c288e-ef1c-445b-a3a0-a253c3433f34 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:46.200365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bfd80-31c9-4bdb-9dc3-540aef8aba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:46.200992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bfd80-31c9-4bdb-9dc3-540aef8aba36 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:46.201267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bfd80-31c9-4bdb-9dc3-540aef8aba36 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:46.201533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bfd80-31c9-4bdb-9dc3-540aef8aba36 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:46.202144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bfd80-31c9-4bdb-9dc3-540aef8aba36 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:46.202441 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bfd80-31c9-4bdb-9dc3-540aef8aba36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:46.204093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bfd80-31c9-4bdb-9dc3-540aef8aba36 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:46.208341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 647/1292] 10.222.0.50 () {56 vars in 1116 bytes} [Thu Sep 15 10:46:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:48.325630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb620f-6ffb-4d94-ab2f-332b9cd9103b None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:48.326282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb620f-6ffb-4d94-ab2f-332b9cd9103b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:48.326282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb620f-6ffb-4d94-ab2f-332b9cd9103b None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:48.327536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 646/1293] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:48.333174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bc0ca-47ef-48ed-a683-0287c7cc9014 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:48.333603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bc0ca-47ef-48ed-a683-0287c7cc9014 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:48.333710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bc0ca-47ef-48ed-a683-0287c7cc9014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:48.351510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6c4bc0ca-47ef-48ed-a683-0287c7cc9014 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:48.352684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6c4bc0ca-47ef-48ed-a683-0287c7cc9014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:48.381475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:48.418588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 648/1294] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:48.689651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb620f-6ffb-4d94-ab2f-332b9cd9103b None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:48.720123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:48.789633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb620f-6ffb-4d94-ab2f-332b9cd9103b None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:48.823217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319290c2-3c5f-4ca1-92d4-a8f10a52bb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:48.823807 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319290c2-3c5f-4ca1-92d4-a8f10a52bb38 None admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:48.824086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319290c2-3c5f-4ca1-92d4-a8f10a52bb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:48.824357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319290c2-3c5f-4ca1-92d4-a8f10a52bb38 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:48.825017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319290c2-3c5f-4ca1-92d4-a8f10a52bb38 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:48.826381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319290c2-3c5f-4ca1-92d4-a8f10a52bb38 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:48.840520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 647/1295] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Sep 15 10:46:48 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:46:50.998403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ea28-35d4-4790-a55a-b43caad42cba None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:50.998959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ea28-35d4-4790-a55a-b43caad42cba None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:50.998959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ea28-35d4-4790-a55a-b43caad42cba None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:51.000503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 649/1296] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:46:51.005863 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143865cf-03d9-4a1f-93c4-2a480fb23395 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:51.006284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143865cf-03d9-4a1f-93c4-2a480fb23395 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:51.006412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143865cf-03d9-4a1f-93c4-2a480fb23395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:51.024735 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-143865cf-03d9-4a1f-93c4-2a480fb23395 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:51.026009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-143865cf-03d9-4a1f-93c4-2a480fb23395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:51.096898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 648/1297] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:51.371761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73ea28-35d4-4790-a55a-b43caad42cba None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:51.470741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73ea28-35d4-4790-a55a-b43caad42cba None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:51.504241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa5301-3ed3-4a8f-a500-92e6e156c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:51.504875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa5301-3ed3-4a8f-a500-92e6e156c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:51.505154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa5301-3ed3-4a8f-a500-92e6e156c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:51.505420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa5301-3ed3-4a8f-a500-92e6e156c8d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:51.505985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa5301-3ed3-4a8f-a500-92e6e156c8d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:51.507346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa5301-3ed3-4a8f-a500-92e6e156c8d7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-50fa5301-3ed3-4a8f-a500-92e6e156c8d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:51.516377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:51.517736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:51.518897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 650/1298] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:46:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:51.523554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143865cf-03d9-4a1f-93c4-2a480fb23395 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:51.541815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:51.622941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143865cf-03d9-4a1f-93c4-2a480fb23395 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:51.655798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065b530-162e-4544-9322-c598d35529bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:51.656434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065b530-162e-4544-9322-c598d35529bb None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:51.656730 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065b530-162e-4544-9322-c598d35529bb None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:51.657031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065b530-162e-4544-9322-c598d35529bb None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:51.657660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065b530-162e-4544-9322-c598d35529bb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:51.657964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065b530-162e-4544-9322-c598d35529bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:51.659333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065b530-162e-4544-9322-c598d35529bb None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:51.664344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 649/1299] 10.222.0.50 () {56 vars in 1116 bytes} [Thu Sep 15 10:46:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:51.670450 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa5301-3ed3-4a8f-a500-92e6e156c8d7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:51.703029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:51.770879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa5301-3ed3-4a8f-a500-92e6e156c8d7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:51.803751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bab7f4-f309-4751-8fcd-a2368b70c75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:51.804347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab7f4-f309-4751-8fcd-a2368b70c75a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:51.804628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab7f4-f309-4751-8fcd-a2368b70c75a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:51.804933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab7f4-f309-4751-8fcd-a2368b70c75a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:51.805624 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bab7f4-f309-4751-8fcd-a2368b70c75a None admin] RBAC: Adding query filter params (interface=public, service_id=5d8fef2e013f4dd1b58e500b5c58ad82, region_id=RegionOne) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:51.805930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bab7f4-f309-4751-8fcd-a2368b70c75a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:51.807305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bab7f4-f309-4751-8fcd-a2368b70c75a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:51.812270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 651/1300] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Sep 15 10:46:51 2022] GET /identity/v3/endpoints?service_id=5d8fef2e013f4dd1b58e500b5c58ad82&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:51.818816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065b530-162e-4544-9322-c598d35529bb None admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:51.897111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:51.914683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065b530-162e-4544-9322-c598d35529bb None admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:51.949122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95227c-e33f-43b6-a425-02d56f1c1723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:51.949714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95227c-e33f-43b6-a425-02d56f1c1723 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:51.949991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95227c-e33f-43b6-a425-02d56f1c1723 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:51.950262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95227c-e33f-43b6-a425-02d56f1c1723 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:51.950861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95227c-e33f-43b6-a425-02d56f1c1723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:51.952214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95227c-e33f-43b6-a425-02d56f1c1723 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:51.958813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 650/1301] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Sep 15 10:46:51 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:46:54.109338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4f00-8bbb-4373-b991-0b5d71d5cb48 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:54.109872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4f00-8bbb-4373-b991-0b5d71d5cb48 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:54.109872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4f00-8bbb-4373-b991-0b5d71d5cb48 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:54.111231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 652/1302] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:46:54.116530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda3c07-09e3-4d61-8111-688e8585767a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:54.116993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda3c07-09e3-4d61-8111-688e8585767a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:54.117101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda3c07-09e3-4d61-8111-688e8585767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:54.135032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9eda3c07-09e3-4d61-8111-688e8585767a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:46:54.136237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9eda3c07-09e3-4d61-8111-688e8585767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:46:54.201887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 651/1303] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:46:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 10:46:54.479465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc4f00-8bbb-4373-b991-0b5d71d5cb48 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:54.573799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:54.581739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc4f00-8bbb-4373-b991-0b5d71d5cb48 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:54.614788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbfd97-4433-44de-8a27-6c53ced08b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:54.615406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbfd97-4433-44de-8a27-6c53ced08b68 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:54.615683 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbfd97-4433-44de-8a27-6c53ced08b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:54.615950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbfd97-4433-44de-8a27-6c53ced08b68 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:54.616513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbfd97-4433-44de-8a27-6c53ced08b68 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:54.617906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbfd97-4433-44de-8a27-6c53ced08b68 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e0dbfd97-4433-44de-8a27-6c53ced08b68 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 10:46:54.627350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 10:46:54.628558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 10:46:54.629687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 653/1304] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Sep 15 10:46:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 10:46:54.634542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda3c07-09e3-4d61-8111-688e8585767a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:54.708209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:46:54.730504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda3c07-09e3-4d61-8111-688e8585767a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:54.763879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ba112-9157-47f3-a8d8-7738128f4302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:54.764469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ba112-9157-47f3-a8d8-7738128f4302 None admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:54.764748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ba112-9157-47f3-a8d8-7738128f4302 None admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:54.765044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ba112-9157-47f3-a8d8-7738128f4302 None admin] PATH_INFO: `/v3/services` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:54.765686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ba112-9157-47f3-a8d8-7738128f4302 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:46:54.765982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ba112-9157-47f3-a8d8-7738128f4302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:54.767377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ba112-9157-47f3-a8d8-7738128f4302 None admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:54.772292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 652/1305] 10.222.0.50 () {56 vars in 1116 bytes} [Thu Sep 15 10:46:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:46:54.778099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbfd97-4433-44de-8a27-6c53ced08b68 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:46:54.877781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbfd97-4433-44de-8a27-6c53ced08b68 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:46:54.910017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a4169-d052-4fdb-95a7-e66942c1baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:46:54.910617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a4169-d052-4fdb-95a7-e66942c1baae None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:46:54.910895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a4169-d052-4fdb-95a7-e66942c1baae None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:46:54.911167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a4169-d052-4fdb-95a7-e66942c1baae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:46:54.911827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a4169-d052-4fdb-95a7-e66942c1baae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:46:54.913222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a4169-d052-4fdb-95a7-e66942c1baae None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:46:54.932647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 654/1306] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:46:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:47:04.769442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=30 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:47:04.928390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:47:59.110309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3478e3-f397-48d4-9328-dc083524d080 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:47:59.110813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3478e3-f397-48d4-9328-dc083524d080 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:47:59.110920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3478e3-f397-48d4-9328-dc083524d080 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:47:59.112375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 653/1307] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:47:59.117923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95594622-31d8-4995-88ea-72a5b9e966cf None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:47:59.118432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95594622-31d8-4995-88ea-72a5b9e966cf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:47:59.118526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95594622-31d8-4995-88ea-72a5b9e966cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:47:59.229190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-95594622-31d8-4995-88ea-72a5b9e966cf None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:47:59.231333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-95594622-31d8-4995-88ea-72a5b9e966cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:47:59.340429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 655/1308] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:47:59.641457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207dedb-4f21-44be-adc8-490d06f477f5 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:47:59.641790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207dedb-4f21-44be-adc8-490d06f477f5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:47:59.642015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207dedb-4f21-44be-adc8-490d06f477f5 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:47:59.643461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 654/1309] 10.222.0.50 () {54 vars in 965 bytes} [Thu Sep 15 10:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:47:59.650079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839692e-a6ce-4ded-baaa-e417f9895cfd None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:47:59.650435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839692e-a6ce-4ded-baaa-e417f9895cfd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:47:59.650838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839692e-a6ce-4ded-baaa-e417f9895cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:47:59.737856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c839692e-a6ce-4ded-baaa-e417f9895cfd None None] MFA Rules not processed for user `f5e7b08dc3544e76ac6732ca2aa89bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:47:59.739158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c839692e-a6ce-4ded-baaa-e417f9895cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:47:59.761511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:47:59.821266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 656/1310] 10.222.0.50 () {58 vars in 1062 bytes} [Thu Sep 15 10:47:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:47:59.830236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207dedb-4f21-44be-adc8-490d06f477f5 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:47:59.831963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3207dedb-4f21-44be-adc8-490d06f477f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:47:59.957787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207dedb-4f21-44be-adc8-490d06f477f5 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:00.039887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc102-759b-4a4c-b5a9-7795282f508f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:00.040538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc102-759b-4a4c-b5a9-7795282f508f service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:00.041376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc102-759b-4a4c-b5a9-7795282f508f service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:00.041794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc102-759b-4a4c-b5a9-7795282f508f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:00.043562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cfdfc102-759b-4a4c-b5a9-7795282f508f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:00.104056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:00.163172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfc102-759b-4a4c-b5a9-7795282f508f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:00.164858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfc102-759b-4a4c-b5a9-7795282f508f service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:00.185709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 655/1311] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:02.672616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00058a-0941-4c12-a070-c7764df9a0af None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:02.673167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00058a-0941-4c12-a070-c7764df9a0af None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:02.673167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00058a-0941-4c12-a070-c7764df9a0af None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:02.674456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 657/1312] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:02.679699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d51db-3ad7-485a-bc9b-c3ce3210fae8 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:02.680107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d51db-3ad7-485a-bc9b-c3ce3210fae8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:02.680257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d51db-3ad7-485a-bc9b-c3ce3210fae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:02.768001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7b2d51db-3ad7-485a-bc9b-c3ce3210fae8 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:02.769277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7b2d51db-3ad7-485a-bc9b-c3ce3210fae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:02.867454 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 656/1313] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:03.152384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00058a-0941-4c12-a070-c7764df9a0af None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:03.153945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7b00058a-0941-4c12-a070-c7764df9a0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:03.262627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00058a-0941-4c12-a070-c7764df9a0af None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:03.334004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:03.336282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e58ab1-c24e-4c66-8039-128539f735d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:03.337129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e58ab1-c24e-4c66-8039-128539f735d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:03.338164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e58ab1-c24e-4c66-8039-128539f735d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:03.338405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e58ab1-c24e-4c66-8039-128539f735d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:03.340223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-13e58ab1-c24e-4c66-8039-128539f735d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:03.447674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e58ab1-c24e-4c66-8039-128539f735d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:03.449139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e58ab1-c24e-4c66-8039-128539f735d6 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:03.466856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 658/1314] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:05.919166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299d976-66f2-4d23-98c7-556b483b9b5a None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:05.919690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299d976-66f2-4d23-98c7-556b483b9b5a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:05.919690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299d976-66f2-4d23-98c7-556b483b9b5a None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:05.921077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 657/1315] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:05.926326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f813d6-5fa6-407f-b0bb-dc577b91674e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:05.926746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f813d6-5fa6-407f-b0bb-dc577b91674e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:05.926892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f813d6-5fa6-407f-b0bb-dc577b91674e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:05.993005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-23f813d6-5fa6-407f-b0bb-dc577b91674e None None] MFA Rules not processed for user `c7a4ccc7e7914c20a0f1117adfd425c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:05.994871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-23f813d6-5fa6-407f-b0bb-dc577b91674e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:06.046241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:06.090403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 659/1316] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:05 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:06.400539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd9221-0a02-4144-bc95-4a44bc096ab1 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:06.400869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd9221-0a02-4144-bc95-4a44bc096ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:06.401210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd9221-0a02-4144-bc95-4a44bc096ab1 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:06.402566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 658/1317] 10.222.0.50 () {54 vars in 965 bytes} [Thu Sep 15 10:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:06.408434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb39ec4-d610-46a2-a3fc-b1c3ee19de6d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:06.408732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb39ec4-d610-46a2-a3fc-b1c3ee19de6d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:06.409043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb39ec4-d610-46a2-a3fc-b1c3ee19de6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:06.486257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1fb39ec4-d610-46a2-a3fc-b1c3ee19de6d None None] MFA Rules not processed for user `f5e7b08dc3544e76ac6732ca2aa89bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:06.487472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1fb39ec4-d610-46a2-a3fc-b1c3ee19de6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:06.561746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 660/1318] 10.222.0.50 () {58 vars in 1062 bytes} [Thu Sep 15 10:48:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:06.571243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd9221-0a02-4144-bc95-4a44bc096ab1 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:06.573297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-36fd9221-0a02-4144-bc95-4a44bc096ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:06.582883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:06.679478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd9221-0a02-4144-bc95-4a44bc096ab1 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:06.753252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a369b-fc8f-468e-b3a7-5d1f5024178f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:06.753865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a369b-fc8f-468e-b3a7-5d1f5024178f service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:06.754145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a369b-fc8f-468e-b3a7-5d1f5024178f service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:06.754416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a369b-fc8f-468e-b3a7-5d1f5024178f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:06.756000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-964a369b-fc8f-468e-b3a7-5d1f5024178f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:06.817504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:06.864127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a369b-fc8f-468e-b3a7-5d1f5024178f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:06.865592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a369b-fc8f-468e-b3a7-5d1f5024178f service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:06.883728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 659/1319] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:09.951413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e42cb-a022-4db9-887c-59d66262b64b None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:09.951961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e42cb-a022-4db9-887c-59d66262b64b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:09.951961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e42cb-a022-4db9-887c-59d66262b64b None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:09.953336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 661/1320] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:09.958520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d57ae-eb7b-4224-bc90-168762d2409e None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:09.958956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d57ae-eb7b-4224-bc90-168762d2409e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:09.959056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d57ae-eb7b-4224-bc90-168762d2409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:10.025401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ca8d57ae-eb7b-4224-bc90-168762d2409e None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:10.026640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ca8d57ae-eb7b-4224-bc90-168762d2409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:10.123083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 660/1321] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:10.423900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e42cb-a022-4db9-887c-59d66262b64b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:10.425465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-811e42cb-a022-4db9-887c-59d66262b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:10.475889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:10.528966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e42cb-a022-4db9-887c-59d66262b64b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:10.603847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85738a55-9dfd-480b-805c-4b9e9d1c293a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:10.604494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85738a55-9dfd-480b-805c-4b9e9d1c293a service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:10.604844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85738a55-9dfd-480b-805c-4b9e9d1c293a service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:10.605123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85738a55-9dfd-480b-805c-4b9e9d1c293a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:10.606720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-85738a55-9dfd-480b-805c-4b9e9d1c293a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:10.702539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:10.716370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85738a55-9dfd-480b-805c-4b9e9d1c293a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:10.717842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85738a55-9dfd-480b-805c-4b9e9d1c293a service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:10.735657 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 662/1322] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:13.568510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f388527-c530-4531-b368-338872d510d2 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:13.569021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f388527-c530-4531-b368-338872d510d2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:13.569105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f388527-c530-4531-b368-338872d510d2 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:13.570602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 661/1323] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:13.575974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa128e-9b5e-44c1-a851-6562e3878765 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:13.576430 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa128e-9b5e-44c1-a851-6562e3878765 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:13.576608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa128e-9b5e-44c1-a851-6562e3878765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:13.646265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-96fa128e-9b5e-44c1-a851-6562e3878765 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:13.647622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-96fa128e-9b5e-44c1-a851-6562e3878765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:13.753418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 663/1324] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:14.035394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f388527-c530-4531-b368-338872d510d2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:14.036896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5f388527-c530-4531-b368-338872d510d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:14.057201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:14.144592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f388527-c530-4531-b368-338872d510d2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:14.220457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd21b6-718a-4e26-afe2-857219690094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:14.221097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd21b6-718a-4e26-afe2-857219690094 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:14.221378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd21b6-718a-4e26-afe2-857219690094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:14.221652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd21b6-718a-4e26-afe2-857219690094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:14.223379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5ecd21b6-718a-4e26-afe2-857219690094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:14.297161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:14.336275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd21b6-718a-4e26-afe2-857219690094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:14.337805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd21b6-718a-4e26-afe2-857219690094 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:14.356325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 662/1325] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:16.574548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2c17-5700-40ba-88df-88d8d69bfbef None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:16.574972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2c17-5700-40ba-88df-88d8d69bfbef None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:16.575097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2c17-5700-40ba-88df-88d8d69bfbef None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:16.576607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 664/1326] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:16.581643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaad41b-65bf-46d5-bf6f-b68b98942163 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:16.582032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaad41b-65bf-46d5-bf6f-b68b98942163 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:16.582203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaad41b-65bf-46d5-bf6f-b68b98942163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:16.656771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-aeaad41b-65bf-46d5-bf6f-b68b98942163 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:16.658005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-aeaad41b-65bf-46d5-bf6f-b68b98942163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:16.756461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 663/1327] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:17.044271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf2c17-5700-40ba-88df-88d8d69bfbef None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:17.045829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9dcf2c17-5700-40ba-88df-88d8d69bfbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:17.107078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:17.151571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf2c17-5700-40ba-88df-88d8d69bfbef None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:17.229681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153e4e2-dcbd-4f1d-88d6-477219af3eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:17.230341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153e4e2-dcbd-4f1d-88d6-477219af3eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:17.230587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153e4e2-dcbd-4f1d-88d6-477219af3eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:17.230942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153e4e2-dcbd-4f1d-88d6-477219af3eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:17.232622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9153e4e2-dcbd-4f1d-88d6-477219af3eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:17.340084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:17.342455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153e4e2-dcbd-4f1d-88d6-477219af3eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:17.344231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153e4e2-dcbd-4f1d-88d6-477219af3eb0 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:17.363022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 665/1328] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:19.600179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6f3ff-350d-4a26-803f-a7794d21fa35 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:19.600714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6f3ff-350d-4a26-803f-a7794d21fa35 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:19.600714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6f3ff-350d-4a26-803f-a7794d21fa35 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:19.603223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 664/1329] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:19.608396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c736ff-1e33-4ff8-a4b2-2b70f5cd430f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:19.608837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c736ff-1e33-4ff8-a4b2-2b70f5cd430f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:19.608983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c736ff-1e33-4ff8-a4b2-2b70f5cd430f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:19.681930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-91c736ff-1e33-4ff8-a4b2-2b70f5cd430f None None] MFA Rules not processed for user `c7a4ccc7e7914c20a0f1117adfd425c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:19.683485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-91c736ff-1e33-4ff8-a4b2-2b70f5cd430f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:19.785464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 666/1330] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:19 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:20.077869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6f3ff-350d-4a26-803f-a7794d21fa35 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:20.079445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f5e6f3ff-350d-4a26-803f-a7794d21fa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:20.113149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:20.192479 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6f3ff-350d-4a26-803f-a7794d21fa35 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:20.271949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627908c-2189-4c07-a1b9-7285f0550bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:20.272601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627908c-2189-4c07-a1b9-7285f0550bbf service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:20.273114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627908c-2189-4c07-a1b9-7285f0550bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:20.273398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627908c-2189-4c07-a1b9-7285f0550bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:20.275220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-1627908c-2189-4c07-a1b9-7285f0550bbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:20.356358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:20.388647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627908c-2189-4c07-a1b9-7285f0550bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:20.390155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627908c-2189-4c07-a1b9-7285f0550bbf service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:20.408123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 665/1331] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:22.813502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b6736-d2a3-40d1-bd72-5bcf25eb7353 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:22.814062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b6736-d2a3-40d1-bd72-5bcf25eb7353 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:22.814062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b6736-d2a3-40d1-bd72-5bcf25eb7353 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:22.815570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 667/1332] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:22.821342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bef5e1-92e8-4350-aa4a-66e7697b24f5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:22.821745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bef5e1-92e8-4350-aa4a-66e7697b24f5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:22.821891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bef5e1-92e8-4350-aa4a-66e7697b24f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:22.888891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-80bef5e1-92e8-4350-aa4a-66e7697b24f5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:22.890092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-80bef5e1-92e8-4350-aa4a-66e7697b24f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:22.984715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 666/1333] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:23.274881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b6736-d2a3-40d1-bd72-5bcf25eb7353 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:23.276485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f52b6736-d2a3-40d1-bd72-5bcf25eb7353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:23.346572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:23.380286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b6736-d2a3-40d1-bd72-5bcf25eb7353 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:23.452527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d76b84-30c6-4ac1-abd4-1214395616ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:23.453152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d76b84-30c6-4ac1-abd4-1214395616ea service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:23.453431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d76b84-30c6-4ac1-abd4-1214395616ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:23.453700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d76b84-30c6-4ac1-abd4-1214395616ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:23.455256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-59d76b84-30c6-4ac1-abd4-1214395616ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:23.564828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d76b84-30c6-4ac1-abd4-1214395616ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:23.566303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d76b84-30c6-4ac1-abd4-1214395616ea service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:23.576277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:23.584794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 668/1334] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:25.786024 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116b697-342c-426d-8e82-c78bb6236870 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:25.786611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116b697-342c-426d-8e82-c78bb6236870 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:25.786611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116b697-342c-426d-8e82-c78bb6236870 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:25.787946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 667/1335] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:25.794179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a283b-928b-4a89-afc8-bbf4d9edc6df None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:25.794615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a283b-928b-4a89-afc8-bbf4d9edc6df None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:25.794740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a283b-928b-4a89-afc8-bbf4d9edc6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:25.861243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1e9a283b-928b-4a89-afc8-bbf4d9edc6df None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:25.862385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1e9a283b-928b-4a89-afc8-bbf4d9edc6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:25.958330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 669/1336] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:26.249824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116b697-342c-426d-8e82-c78bb6236870 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:26.251352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4116b697-342c-426d-8e82-c78bb6236870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:26.292928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:26.361111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116b697-342c-426d-8e82-c78bb6236870 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:26.437535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0131b-723c-4661-9d33-3e9e664211c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:26.438143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0131b-723c-4661-9d33-3e9e664211c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:26.438418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0131b-723c-4661-9d33-3e9e664211c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:26.438686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0131b-723c-4661-9d33-3e9e664211c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:26.440526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-43b0131b-723c-4661-9d33-3e9e664211c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:26.529000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:26.559010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0131b-723c-4661-9d33-3e9e664211c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:26.560461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0131b-723c-4661-9d33-3e9e664211c3 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:26.579340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 668/1337] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:29.085352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22c750-77de-40de-81f3-a9f21fb5d244 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:29.085893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22c750-77de-40de-81f3-a9f21fb5d244 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:29.085893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22c750-77de-40de-81f3-a9f21fb5d244 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:29.087229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 670/1338] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:29.092426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66bf16-e073-4b25-80f5-68407b41f8a1 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:29.092864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66bf16-e073-4b25-80f5-68407b41f8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:29.093024 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66bf16-e073-4b25-80f5-68407b41f8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:29.158713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7a66bf16-e073-4b25-80f5-68407b41f8a1 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:29.159906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7a66bf16-e073-4b25-80f5-68407b41f8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:29.259555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 669/1339] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:29.547340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22c750-77de-40de-81f3-a9f21fb5d244 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:29.548907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5b22c750-77de-40de-81f3-a9f21fb5d244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:29.625260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:29.653405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22c750-77de-40de-81f3-a9f21fb5d244 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:29.726051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c0219-194f-4b8d-8aa4-98330b505015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:29.726800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c0219-194f-4b8d-8aa4-98330b505015 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:29.727071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c0219-194f-4b8d-8aa4-98330b505015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:29.727489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c0219-194f-4b8d-8aa4-98330b505015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:29.729126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8c8c0219-194f-4b8d-8aa4-98330b505015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:29.837507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c0219-194f-4b8d-8aa4-98330b505015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:29.838853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c0219-194f-4b8d-8aa4-98330b505015 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:29.857088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 671/1340] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:34.170655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a9246-a905-4ab0-a068-eda9816977c8 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:34.171169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a9246-a905-4ab0-a068-eda9816977c8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:34.171342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a9246-a905-4ab0-a068-eda9816977c8 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:34.172751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 670/1341] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:34.179217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddab09c-81af-4e8e-86e1-7aa2e7c8f45c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:34.181901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddab09c-81af-4e8e-86e1-7aa2e7c8f45c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:34.182197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddab09c-81af-4e8e-86e1-7aa2e7c8f45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:34.190266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:34.266443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0ddab09c-81af-4e8e-86e1-7aa2e7c8f45c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:34.267683 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0ddab09c-81af-4e8e-86e1-7aa2e7c8f45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:34.381603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 672/1342] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:48:34.676673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a9246-a905-4ab0-a068-eda9816977c8 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:34.678307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-274a9246-a905-4ab0-a068-eda9816977c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:34.732738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:34.803538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a9246-a905-4ab0-a068-eda9816977c8 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:34.891811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac12564-e635-43dd-8798-51d87c281b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:34.892878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12564-e635-43dd-8798-51d87c281b7e service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:34.893376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12564-e635-43dd-8798-51d87c281b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:34.893845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12564-e635-43dd-8798-51d87c281b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:34.896916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3ac12564-e635-43dd-8798-51d87c281b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:35.002685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:35.020395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac12564-e635-43dd-8798-51d87c281b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:35.021901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac12564-e635-43dd-8798-51d87c281b7e service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:35.041856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 671/1343] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:37.855628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5dcf7-8a76-41f0-a41a-5b137d486525 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:37.856213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5dcf7-8a76-41f0-a41a-5b137d486525 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:37.856213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5dcf7-8a76-41f0-a41a-5b137d486525 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:37.857701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 673/1344] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:37.862784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70d5ae-d009-4849-9ee5-404688bc54b2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:37.863180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70d5ae-d009-4849-9ee5-404688bc54b2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:37.863502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70d5ae-d009-4849-9ee5-404688bc54b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:37.930667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-cd70d5ae-d009-4849-9ee5-404688bc54b2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:37.931887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cd70d5ae-d009-4849-9ee5-404688bc54b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:38.030844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 672/1345] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:38.317066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5dcf7-8a76-41f0-a41a-5b137d486525 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:38.318607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1ed5dcf7-8a76-41f0-a41a-5b137d486525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:38.410061 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:38.430242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5dcf7-8a76-41f0-a41a-5b137d486525 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:38.506118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a035d6-c7ce-44bc-a561-cc48fdd15c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:38.506736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a035d6-c7ce-44bc-a561-cc48fdd15c92 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:38.507017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a035d6-c7ce-44bc-a561-cc48fdd15c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:38.507339 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a035d6-c7ce-44bc-a561-cc48fdd15c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:38.508954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-34a035d6-c7ce-44bc-a561-cc48fdd15c92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:38.623251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a035d6-c7ce-44bc-a561-cc48fdd15c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:38.624835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a035d6-c7ce-44bc-a561-cc48fdd15c92 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:38.643063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 674/1346] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:43.060396 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6533f-8c84-4f1a-8602-e76beef4e12d None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:43.060883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6533f-8c84-4f1a-8602-e76beef4e12d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:43.060980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6533f-8c84-4f1a-8602-e76beef4e12d None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:43.062351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 673/1347] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:43.068027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895569d7-3787-43d0-bb18-5846180bab49 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:43.068524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895569d7-3787-43d0-bb18-5846180bab49 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:43.068694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895569d7-3787-43d0-bb18-5846180bab49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:43.086527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:43.140703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-895569d7-3787-43d0-bb18-5846180bab49 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:43.141950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-895569d7-3787-43d0-bb18-5846180bab49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:43.241718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 675/1348] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:43.526660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6533f-8c84-4f1a-8602-e76beef4e12d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:43.528181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f9a6533f-8c84-4f1a-8602-e76beef4e12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:43.587486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:43.634701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6533f-8c84-4f1a-8602-e76beef4e12d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:43.710756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bffa1c-7fdc-44f1-b97d-a7598313721c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:43.711358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bffa1c-7fdc-44f1-b97d-a7598313721c service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:43.711637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bffa1c-7fdc-44f1-b97d-a7598313721c service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:43.711906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bffa1c-7fdc-44f1-b97d-a7598313721c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:43.713547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-80bffa1c-7fdc-44f1-b97d-a7598313721c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:43.817879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:43.822828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bffa1c-7fdc-44f1-b97d-a7598313721c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:43.824279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bffa1c-7fdc-44f1-b97d-a7598313721c service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:43.842833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 674/1349] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:48:48.805945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5389bb-17cd-4aaf-a8b3-70cb86b40ee5 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:48.806411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5389bb-17cd-4aaf-a8b3-70cb86b40ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:48.806847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5389bb-17cd-4aaf-a8b3-70cb86b40ee5 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:48.808688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 676/1350] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 10:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:48.814329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aee5d-43fe-475d-823e-8b45969ae947 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:48.814785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aee5d-43fe-475d-823e-8b45969ae947 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:48.815096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aee5d-43fe-475d-823e-8b45969ae947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:48.995173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-109aee5d-43fe-475d-823e-8b45969ae947 None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:48.996735 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-109aee5d-43fe-475d-823e-8b45969ae947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:49.125911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 675/1351] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:48:53.239914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=59 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:53.295134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36b9d-89fe-4fae-889c-2821ef052101 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:53.295758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36b9d-89fe-4fae-889c-2821ef052101 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:53.296147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36b9d-89fe-4fae-889c-2821ef052101 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:53.297733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 677/1352] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 10:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:53.302661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42767d-7776-4f1c-b26b-1b7c4a95d60d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:53.303773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42767d-7776-4f1c-b26b-1b7c4a95d60d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:53.303773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42767d-7776-4f1c-b26b-1b7c4a95d60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:53.361300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:53.386458 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ee42767d-7776-4f1c-b26b-1b7c4a95d60d None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:53.387677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ee42767d-7776-4f1c-b26b-1b7c4a95d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:53.502575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 676/1353] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 10:48:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:54.942617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5452577-18e2-4ed5-bd57-64792357c31b None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:54.943167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5452577-18e2-4ed5-bd57-64792357c31b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:54.943167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5452577-18e2-4ed5-bd57-64792357c31b None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:54.945150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 678/1354] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 10:48:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:48:54.950178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0160a5a-d9fb-4e18-8784-4c62fc0d6790 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:54.950630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0160a5a-d9fb-4e18-8784-4c62fc0d6790 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:54.950897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0160a5a-d9fb-4e18-8784-4c62fc0d6790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:55.031031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-c0160a5a-d9fb-4e18-8784-4c62fc0d6790 None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:55.032280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c0160a5a-d9fb-4e18-8784-4c62fc0d6790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:55.074310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:55.140761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 677/1355] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 10:48:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:58.643276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93092e0f-b1e8-46f2-81fa-7c30278ba904 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:58.643802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93092e0f-b1e8-46f2-81fa-7c30278ba904 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:58.643802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93092e0f-b1e8-46f2-81fa-7c30278ba904 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:58.645363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 679/1356] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:58.651599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a522cd-4bda-455b-9dd4-287cb547693f None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:58.652383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a522cd-4bda-455b-9dd4-287cb547693f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:58.652852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a522cd-4bda-455b-9dd4-287cb547693f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:58.730758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-b3a522cd-4bda-455b-9dd4-287cb547693f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:58.732346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b3a522cd-4bda-455b-9dd4-287cb547693f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:58.839471 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 678/1357] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:59.148359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33579379-f786-4d62-8294-06a2169899e1 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:59.149223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33579379-f786-4d62-8294-06a2169899e1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:59.150138 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33579379-f786-4d62-8294-06a2169899e1 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:59.154179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 680/1358] 10.222.0.50 () {54 vars in 962 bytes} [Thu Sep 15 10:48:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:48:59.163514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b92ff-6349-4729-8903-bf436687e588 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:59.165207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b92ff-6349-4729-8903-bf436687e588 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:59.166112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b92ff-6349-4729-8903-bf436687e588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:59.194877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:59.262206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d47b92ff-6349-4729-8903-bf436687e588 None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:48:59.263506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d47b92ff-6349-4729-8903-bf436687e588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:59.390714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 679/1359] 10.222.0.50 () {58 vars in 1059 bytes} [Thu Sep 15 10:48:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:48:59.398955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33579379-f786-4d62-8294-06a2169899e1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:48:59.400573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-33579379-f786-4d62-8294-06a2169899e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:59.563081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33579379-f786-4d62-8294-06a2169899e1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:48:59.646214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:48:59.689618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b72e85-ddb8-4432-9f9f-87834e60eb25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:48:59.690233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b72e85-ddb8-4432-9f9f-87834e60eb25 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:48:59.690515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b72e85-ddb8-4432-9f9f-87834e60eb25 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:48:59.690789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b72e85-ddb8-4432-9f9f-87834e60eb25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:48:59.692517 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-14b72e85-ddb8-4432-9f9f-87834e60eb25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:48:59.798912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b72e85-ddb8-4432-9f9f-87834e60eb25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:48:59.800685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b72e85-ddb8-4432-9f9f-87834e60eb25 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:48:59.821077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 681/1360] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:02.247205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da313187-8f19-4c01-a05a-71625ccd2765 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:02.247740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da313187-8f19-4c01-a05a-71625ccd2765 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:02.247740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da313187-8f19-4c01-a05a-71625ccd2765 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:02.249425 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 680/1361] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:02.254152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ccd7c-83dc-4e4d-843f-a03ae900702a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:02.254547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ccd7c-83dc-4e4d-843f-a03ae900702a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:02.254887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ccd7c-83dc-4e4d-843f-a03ae900702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:02.325031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0e3ccd7c-83dc-4e4d-843f-a03ae900702a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:02.326244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0e3ccd7c-83dc-4e4d-843f-a03ae900702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:02.370785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:02.447982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 682/1362] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:02.733457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da313187-8f19-4c01-a05a-71625ccd2765 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:02.734974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-da313187-8f19-4c01-a05a-71625ccd2765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:02.809301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:02.914512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da313187-8f19-4c01-a05a-71625ccd2765 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:03.054275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2c2ac-2fa1-4081-b368-63020f4be829 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:03.054892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2c2ac-2fa1-4081-b368-63020f4be829 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:03.055172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2c2ac-2fa1-4081-b368-63020f4be829 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:03.055441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2c2ac-2fa1-4081-b368-63020f4be829 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:03.057160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-36b2c2ac-2fa1-4081-b368-63020f4be829 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:03.082194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:03.155594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2c2ac-2fa1-4081-b368-63020f4be829 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:03.157254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2c2ac-2fa1-4081-b368-63020f4be829 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:03.175994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 681/1363] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:05.713135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d7d99-fc3b-4432-a3aa-22e02431bb82 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:05.713706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d7d99-fc3b-4432-a3aa-22e02431bb82 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:05.713706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d7d99-fc3b-4432-a3aa-22e02431bb82 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:05.715020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 683/1364] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:05.720537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91a220-25a1-4e4c-b35d-2ada65a371f0 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:05.721014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91a220-25a1-4e4c-b35d-2ada65a371f0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:05.721318 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91a220-25a1-4e4c-b35d-2ada65a371f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:05.796625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7f91a220-25a1-4e4c-b35d-2ada65a371f0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:05.798060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7f91a220-25a1-4e4c-b35d-2ada65a371f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:05.908232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:05.910906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 682/1365] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:06.217208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5acc9d-52c0-4d54-8756-9c780a2b1015 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:06.217499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5acc9d-52c0-4d54-8756-9c780a2b1015 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:06.217774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5acc9d-52c0-4d54-8756-9c780a2b1015 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:06.219288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 684/1366] 10.222.0.50 () {54 vars in 962 bytes} [Thu Sep 15 10:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:06.225550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e4e9e-c9b9-4277-be65-7f30df6297ba None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:06.225942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e4e9e-c9b9-4277-be65-7f30df6297ba None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:06.226317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e4e9e-c9b9-4277-be65-7f30df6297ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:06.309158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9c1e4e9e-c9b9-4277-be65-7f30df6297ba None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:06.310554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9c1e4e9e-c9b9-4277-be65-7f30df6297ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:06.437848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 683/1367] 10.222.0.50 () {58 vars in 1059 bytes} [Thu Sep 15 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:06.445800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5acc9d-52c0-4d54-8756-9c780a2b1015 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:06.447505 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5c5acc9d-52c0-4d54-8756-9c780a2b1015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:06.617202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5acc9d-52c0-4d54-8756-9c780a2b1015 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:06.640920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:06.742568 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90067837-da2c-4b8b-b66a-b54e87fce07d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:06.743212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90067837-da2c-4b8b-b66a-b54e87fce07d service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:06.743587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90067837-da2c-4b8b-b66a-b54e87fce07d service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:06.743959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90067837-da2c-4b8b-b66a-b54e87fce07d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:06.745697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-90067837-da2c-4b8b-b66a-b54e87fce07d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:06.849049 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90067837-da2c-4b8b-b66a-b54e87fce07d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:06.850842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90067837-da2c-4b8b-b66a-b54e87fce07d service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:06.870204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 685/1368] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:06.880648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790e39d-bf87-4233-a6d8-3a5d80097248 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:06.881269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790e39d-bf87-4233-a6d8-3a5d80097248 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:06.881269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790e39d-bf87-4233-a6d8-3a5d80097248 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:06.882557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 684/1369] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:06.887756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc8f6a-9400-4298-9a51-ae0813e18ee6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:06.888064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc8f6a-9400-4298-9a51-ae0813e18ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:06.888318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc8f6a-9400-4298-9a51-ae0813e18ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:06.922827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:06.970831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-80dc8f6a-9400-4298-9a51-ae0813e18ee6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:06.972205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-80dc8f6a-9400-4298-9a51-ae0813e18ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:07.104929 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 686/1370] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:07.439405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4685e-cd80-4f05-b5e0-cad787f5c301 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:07.439405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4685e-cd80-4f05-b5e0-cad787f5c301 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:07.439405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4685e-cd80-4f05-b5e0-cad787f5c301 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:07.441552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 685/1371] 10.222.0.50 () {54 vars in 964 bytes} [Thu Sep 15 10:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:07.445482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec64abb-e75e-4966-b9de-84c85402efc7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:07.445768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec64abb-e75e-4966-b9de-84c85402efc7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:07.446042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec64abb-e75e-4966-b9de-84c85402efc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:07.524419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3ec64abb-e75e-4966-b9de-84c85402efc7 None None] MFA Rules not processed for user `d7fa5d550a2d442a820cf4e12ebc36c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:07.525795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3ec64abb-e75e-4966-b9de-84c85402efc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:07.585428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:07.619342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 687/1372] 10.222.0.50 () {58 vars in 1061 bytes} [Thu Sep 15 10:49:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:07.626920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4685e-cd80-4f05-b5e0-cad787f5c301 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:07.628408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-65d4685e-cd80-4f05-b5e0-cad787f5c301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:07.682466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:07.741084 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4685e-cd80-4f05-b5e0-cad787f5c301 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:07.831827 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fb60c-c6a7-486a-a696-8525ed40516d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:07.834025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fb60c-c6a7-486a-a696-8525ed40516d service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:07.834321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fb60c-c6a7-486a-a696-8525ed40516d service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:07.834605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fb60c-c6a7-486a-a696-8525ed40516d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:07.836237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-df8fb60c-c6a7-486a-a696-8525ed40516d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:07.987792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:08.008290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fb60c-c6a7-486a-a696-8525ed40516d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:08.010171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fb60c-c6a7-486a-a696-8525ed40516d service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:08.040896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 686/1373] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:09.142853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf62f8-b65a-4b03-8bc5-133c825f04b8 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:09.143385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf62f8-b65a-4b03-8bc5-133c825f04b8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:09.143385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf62f8-b65a-4b03-8bc5-133c825f04b8 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:09.144788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 688/1374] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:09.150830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d4a22-0b51-42b1-911a-7ab79ba6d403 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:09.151279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d4a22-0b51-42b1-911a-7ab79ba6d403 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:09.151408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d4a22-0b51-42b1-911a-7ab79ba6d403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:09.219222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-547d4a22-0b51-42b1-911a-7ab79ba6d403 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:09.220419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-547d4a22-0b51-42b1-911a-7ab79ba6d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:09.321980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 687/1375] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:09.617253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf62f8-b65a-4b03-8bc5-133c825f04b8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:09.618812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b6bf62f8-b65a-4b03-8bc5-133c825f04b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:09.778879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf62f8-b65a-4b03-8bc5-133c825f04b8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:09.834579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:09.899515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf364e2-8187-42ca-97aa-4c309e22fcea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:09.900137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf364e2-8187-42ca-97aa-4c309e22fcea service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:09.900416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf364e2-8187-42ca-97aa-4c309e22fcea service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:09.900689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf364e2-8187-42ca-97aa-4c309e22fcea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:09.902324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3bf364e2-8187-42ca-97aa-4c309e22fcea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:09.996592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf364e2-8187-42ca-97aa-4c309e22fcea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:09.998336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf364e2-8187-42ca-97aa-4c309e22fcea service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:10.016691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 689/1376] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:12.278605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa2c0-dc4d-49b9-9e7d-aa383b2534a4 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:12.279154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa2c0-dc4d-49b9-9e7d-aa383b2534a4 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:12.279154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa2c0-dc4d-49b9-9e7d-aa383b2534a4 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:12.282572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 688/1377] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:12.286621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a97c2-cc8a-4d38-b3d2-d4f0d5551d06 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:12.287221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a97c2-cc8a-4d38-b3d2-d4f0d5551d06 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:12.287221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a97c2-cc8a-4d38-b3d2-d4f0d5551d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:12.355593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9c2a97c2-cc8a-4d38-b3d2-d4f0d5551d06 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:12.356829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9c2a97c2-cc8a-4d38-b3d2-d4f0d5551d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:12.365363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:12.467258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 690/1378] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:12.758059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffa2c0-dc4d-49b9-9e7d-aa383b2534a4 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:12.759668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-beffa2c0-dc4d-49b9-9e7d-aa383b2534a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:12.826630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:12.918262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffa2c0-dc4d-49b9-9e7d-aa383b2534a4 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:13.039094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c47eee-c465-4be2-8495-430532126b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:13.039694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c47eee-c465-4be2-8495-430532126b0a service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:13.039969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c47eee-c465-4be2-8495-430532126b0a service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:13.040233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c47eee-c465-4be2-8495-430532126b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:13.041817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-42c47eee-c465-4be2-8495-430532126b0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:13.062274 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:13.143011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c47eee-c465-4be2-8495-430532126b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:13.144622 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c47eee-c465-4be2-8495-430532126b0a service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:13.164791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 689/1379] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:15.437947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a7b34-99cf-4af0-ba66-e15252e763c2 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:15.438439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a7b34-99cf-4af0-ba66-e15252e763c2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:15.438547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a7b34-99cf-4af0-ba66-e15252e763c2 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:15.439834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 691/1380] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:15.445868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64deee3-1e97-481a-b9b8-9434abaaeab8 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:15.446279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64deee3-1e97-481a-b9b8-9434abaaeab8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:15.446478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64deee3-1e97-481a-b9b8-9434abaaeab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:15.546234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-b64deee3-1e97-481a-b9b8-9434abaaeab8 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:15.548247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b64deee3-1e97-481a-b9b8-9434abaaeab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:15.675576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:15.682824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 690/1381] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:15.981229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a7b34-99cf-4af0-ba66-e15252e763c2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:15.982863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b17a7b34-99cf-4af0-ba66-e15252e763c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:16.125360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:16.136647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a7b34-99cf-4af0-ba66-e15252e763c2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:16.261028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71a670-325f-43ce-96fa-df80db7b0650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:16.261609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a670-325f-43ce-96fa-df80db7b0650 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:16.261898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a670-325f-43ce-96fa-df80db7b0650 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:16.262172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a670-325f-43ce-96fa-df80db7b0650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:16.263834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ad71a670-325f-43ce-96fa-df80db7b0650 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:16.367048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71a670-325f-43ce-96fa-df80db7b0650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:16.369391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71a670-325f-43ce-96fa-df80db7b0650 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:16.383153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:16.394238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 692/1382] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:18.732925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2c207-4f88-4ebd-a621-66bc5304b512 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:18.733617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2c207-4f88-4ebd-a621-66bc5304b512 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:18.733617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2c207-4f88-4ebd-a621-66bc5304b512 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:18.734903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 691/1383] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:18.741163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277af64-c53c-48d9-853e-90c4eaaa0077 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:18.741628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277af64-c53c-48d9-853e-90c4eaaa0077 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:18.741832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277af64-c53c-48d9-853e-90c4eaaa0077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:18.822923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9277af64-c53c-48d9-853e-90c4eaaa0077 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:18.824179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9277af64-c53c-48d9-853e-90c4eaaa0077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:18.949645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 693/1384] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:19.240474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2c207-4f88-4ebd-a621-66bc5304b512 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:19.242049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7ae2c207-4f88-4ebd-a621-66bc5304b512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:19.395711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2c207-4f88-4ebd-a621-66bc5304b512 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:19.467091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:19.527508 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b001605-5c5c-46ad-8975-0975700e277d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:19.528224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001605-5c5c-46ad-8975-0975700e277d service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:19.528568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001605-5c5c-46ad-8975-0975700e277d service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:19.528925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001605-5c5c-46ad-8975-0975700e277d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:19.530951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8b001605-5c5c-46ad-8975-0975700e277d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:19.638843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b001605-5c5c-46ad-8975-0975700e277d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:19.640991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b001605-5c5c-46ad-8975-0975700e277d service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:19.661903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 692/1385] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:22.140403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3690a-51a6-49cb-aaa7-2283bb7a19ff None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:22.141033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3690a-51a6-49cb-aaa7-2283bb7a19ff None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:22.141033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3690a-51a6-49cb-aaa7-2283bb7a19ff None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:22.142319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 694/1386] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:22.148178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d86d1-d741-4099-ac93-4c7b40b91dcb None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:22.148625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d86d1-d741-4099-ac93-4c7b40b91dcb None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:22.148730 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d86d1-d741-4099-ac93-4c7b40b91dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:22.215932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9f4d86d1-d741-4099-ac93-4c7b40b91dcb None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:22.217302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9f4d86d1-d741-4099-ac93-4c7b40b91dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:22.251093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:22.315876 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 693/1387] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:22.603552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3690a-51a6-49cb-aaa7-2283bb7a19ff None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:22.605082 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fcc3690a-51a6-49cb-aaa7-2283bb7a19ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:22.682762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:22.754957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3690a-51a6-49cb-aaa7-2283bb7a19ff None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:22.876592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dfda3-b2bb-44ad-b6ed-6d014913716b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:22.877507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dfda3-b2bb-44ad-b6ed-6d014913716b service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:22.877998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dfda3-b2bb-44ad-b6ed-6d014913716b service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:22.878483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dfda3-b2bb-44ad-b6ed-6d014913716b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:22.881101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-963dfda3-b2bb-44ad-b6ed-6d014913716b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:22.922659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:22.979379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dfda3-b2bb-44ad-b6ed-6d014913716b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:22.980952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dfda3-b2bb-44ad-b6ed-6d014913716b service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:22.999457 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 695/1388] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:23.393657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e9fd4-9aec-444c-a5ae-08516420b8ec None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:23.394183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e9fd4-9aec-444c-a5ae-08516420b8ec None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:23.394183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e9fd4-9aec-444c-a5ae-08516420b8ec None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:23.395992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 694/1389] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:23.401434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e86ef-07f5-4131-b91f-ea5450c47898 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:23.401974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e86ef-07f5-4131-b91f-ea5450c47898 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:23.402074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e86ef-07f5-4131-b91f-ea5450c47898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:23.469719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c90e86ef-07f5-4131-b91f-ea5450c47898 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:23.471079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c90e86ef-07f5-4131-b91f-ea5450c47898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:23.575933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 696/1390] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:23.881021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb901aa-5f21-46d7-9071-f954dca6799a None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:23.881312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb901aa-5f21-46d7-9071-f954dca6799a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:23.881590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb901aa-5f21-46d7-9071-f954dca6799a None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:23.882940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 695/1391] 10.222.0.50 () {54 vars in 962 bytes} [Thu Sep 15 10:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:23.888359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8cd8b-e886-4c40-a2fd-fede99f38452 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:23.888717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8cd8b-e886-4c40-a2fd-fede99f38452 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:23.889091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8cd8b-e886-4c40-a2fd-fede99f38452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:23.905355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:23.970818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f5f8cd8b-e886-4c40-a2fd-fede99f38452 None None] MFA Rules not processed for user `6908a326a3d3437c9db898976a2def1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:23.972460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f5f8cd8b-e886-4c40-a2fd-fede99f38452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:24.047895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 697/1392] 10.222.0.50 () {58 vars in 1059 bytes} [Thu Sep 15 10:49:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:24.057012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb901aa-5f21-46d7-9071-f954dca6799a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:24.058576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4fb901aa-5f21-46d7-9071-f954dca6799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:24.165122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb901aa-5f21-46d7-9071-f954dca6799a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:24.224724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:24.240833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739497b-7461-4eff-9407-8a1fb68ec572 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:24.241455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739497b-7461-4eff-9407-8a1fb68ec572 service glance] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:24.241733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739497b-7461-4eff-9407-8a1fb68ec572 service glance] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:24.242004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739497b-7461-4eff-9407-8a1fb68ec572 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:24.243589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8739497b-7461-4eff-9407-8a1fb68ec572 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:24.360003 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739497b-7461-4eff-9407-8a1fb68ec572 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:24.361524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739497b-7461-4eff-9407-8a1fb68ec572 service glance] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:24.381759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 696/1393] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:25.132563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcb3d1-5085-4768-9d5a-81cc2e1e714e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:25.133206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcb3d1-5085-4768-9d5a-81cc2e1e714e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:25.133379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcb3d1-5085-4768-9d5a-81cc2e1e714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:25.203648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-66bcb3d1-5085-4768-9d5a-81cc2e1e714e None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:25.204877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-66bcb3d1-5085-4768-9d5a-81cc2e1e714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:25.243718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832df50-c878-4581-8375-d34ed4b000e8 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:25.244279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832df50-c878-4581-8375-d34ed4b000e8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:25.244279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832df50-c878-4581-8375-d34ed4b000e8 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:25.246362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 697/1394] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:49:25.251644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ad6ac-a0e3-428e-996d-290c06453014 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:25.251933 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ad6ac-a0e3-428e-996d-290c06453014 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:25.252212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ad6ac-a0e3-428e-996d-290c06453014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:25.264493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:25.298297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 698/1395] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 10:49:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:25.325198 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-f52ad6ac-a0e3-428e-996d-290c06453014 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:25.326417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f52ad6ac-a0e3-428e-996d-290c06453014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:25.349038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65302-c005-48fd-8848-474b006443e7 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:25.349303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65302-c005-48fd-8848-474b006443e7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:25.349648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65302-c005-48fd-8848-474b006443e7 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:25.351326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 699/1396] 10.222.0.50 () {54 vars in 943 bytes} [Thu Sep 15 10:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:49:25.357150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac1788-6002-463e-82d7-c75a9e4659f4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:25.357432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac1788-6002-463e-82d7-c75a9e4659f4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:25.357704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac1788-6002-463e-82d7-c75a9e4659f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:25.371648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:25.428411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 698/1397] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:49:25.437837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-40ac1788-6002-463e-82d7-c75a9e4659f4 None None] MFA Rules not processed for user `06007b3b2260409f863ac3d0ac85f4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:25.439060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-40ac1788-6002-463e-82d7-c75a9e4659f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:25.545062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 700/1398] 10.222.0.50 () {58 vars in 1040 bytes} [Thu Sep 15 10:49:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:49:25.556771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ad6ac-a0e3-428e-996d-290c06453014 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:25.558013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f52ad6ac-a0e3-428e-996d-290c06453014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:25.662899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ad6ac-a0e3-428e-996d-290c06453014 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:25.722613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac1788-6002-463e-82d7-c75a9e4659f4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:25.723965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-40ac1788-6002-463e-82d7-c75a9e4659f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:25.737880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:25.757851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:25.782979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ff318-7bfb-4d7e-8d45-e941218ada0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:25.783705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ff318-7bfb-4d7e-8d45-e941218ada0f service swift] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:25.783989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ff318-7bfb-4d7e-8d45-e941218ada0f service swift] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:25.784262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ff318-7bfb-4d7e-8d45-e941218ada0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:25.785999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cb8ff318-7bfb-4d7e-8d45-e941218ada0f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:25.829338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac1788-6002-463e-82d7-c75a9e4659f4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:25.872542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ff318-7bfb-4d7e-8d45-e941218ada0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:25.874180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ff318-7bfb-4d7e-8d45-e941218ada0f service swift] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:25.883125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 699/1399] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 10:49:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 10:49:25.950297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86fc89-cf1e-402a-a4be-62a1c55efe4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:25.951001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86fc89-cf1e-402a-a4be-62a1c55efe4d service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:25.951276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86fc89-cf1e-402a-a4be-62a1c55efe4d service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:25.951583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86fc89-cf1e-402a-a4be-62a1c55efe4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:25.953206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ba86fc89-cf1e-402a-a4be-62a1c55efe4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:26.002476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:26.061725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86fc89-cf1e-402a-a4be-62a1c55efe4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:26.063840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86fc89-cf1e-402a-a4be-62a1c55efe4d service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:26.085042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 701/1400] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:49:28.348692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9f552-a83a-429f-9a91-1f840a4d325d None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:28.349295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9f552-a83a-429f-9a91-1f840a4d325d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:28.349295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9f552-a83a-429f-9a91-1f840a4d325d None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:28.350996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 700/1401] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:28.356227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336bf037-1f9a-48e8-bd85-23def91a8815 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:28.356654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336bf037-1f9a-48e8-bd85-23def91a8815 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:28.356783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336bf037-1f9a-48e8-bd85-23def91a8815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:28.429578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-336bf037-1f9a-48e8-bd85-23def91a8815 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:28.430802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-336bf037-1f9a-48e8-bd85-23def91a8815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:28.560906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 702/1402] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:28.849461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9f552-a83a-429f-9a91-1f840a4d325d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:28.851047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-76a9f552-a83a-429f-9a91-1f840a4d325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:28.963239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:29.035409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9f552-a83a-429f-9a91-1f840a4d325d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:29.163791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbfc80-f2cb-460b-8870-deeb0361cd86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:29.164405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbfc80-f2cb-460b-8870-deeb0361cd86 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:29.164684 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbfc80-f2cb-460b-8870-deeb0361cd86 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:29.164991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbfc80-f2cb-460b-8870-deeb0361cd86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:29.168280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-bbfbfc80-f2cb-460b-8870-deeb0361cd86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:29.248353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:29.284488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbfc80-f2cb-460b-8870-deeb0361cd86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:29.286814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbfc80-f2cb-460b-8870-deeb0361cd86 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:29.306950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 701/1403] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:31.574682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689187-b834-42f5-828b-f16676b1ff2e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:31.575242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689187-b834-42f5-828b-f16676b1ff2e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:31.575242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689187-b834-42f5-828b-f16676b1ff2e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:31.576578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 703/1404] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:49:31.582138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82e17a-3ea5-4938-b46e-ac0ea1afecd7 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:31.583515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82e17a-3ea5-4938-b46e-ac0ea1afecd7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:31.583825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82e17a-3ea5-4938-b46e-ac0ea1afecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:31.659102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9a82e17a-3ea5-4938-b46e-ac0ea1afecd7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:31.660324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9a82e17a-3ea5-4938-b46e-ac0ea1afecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:31.781012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 702/1405] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:32.077098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689187-b834-42f5-828b-f16676b1ff2e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:32.078646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1d689187-b834-42f5-828b-f16676b1ff2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:32.103495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:32.252823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689187-b834-42f5-828b-f16676b1ff2e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:32.361047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:32.388420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27125968-6d90-470c-9dcc-f39002db1cd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:32.389237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27125968-6d90-470c-9dcc-f39002db1cd5 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:32.389510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27125968-6d90-470c-9dcc-f39002db1cd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:32.389787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27125968-6d90-470c-9dcc-f39002db1cd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:32.391435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-27125968-6d90-470c-9dcc-f39002db1cd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:32.503198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27125968-6d90-470c-9dcc-f39002db1cd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:32.504879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27125968-6d90-470c-9dcc-f39002db1cd5 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:32.524601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 704/1406] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:34.811204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd2908-48a7-429c-a34f-06f102d2d763 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:34.811763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd2908-48a7-429c-a34f-06f102d2d763 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:34.811763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd2908-48a7-429c-a34f-06f102d2d763 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:34.813141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 703/1407] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:34.819270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e794d-4d09-426e-88f4-9564b9a3446f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:34.819701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e794d-4d09-426e-88f4-9564b9a3446f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:34.819850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e794d-4d09-426e-88f4-9564b9a3446f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:34.893546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-537e794d-4d09-426e-88f4-9564b9a3446f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:34.894764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-537e794d-4d09-426e-88f4-9564b9a3446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:34.960243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:35.012758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 705/1408] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:35.305631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd2908-48a7-429c-a34f-06f102d2d763 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:35.307173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-51fd2908-48a7-429c-a34f-06f102d2d763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:35.357935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:35.478601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd2908-48a7-429c-a34f-06f102d2d763 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:35.615867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd34e3-3afa-4375-8b5f-b053b2004be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:35.616464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd34e3-3afa-4375-8b5f-b053b2004be0 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:35.616738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd34e3-3afa-4375-8b5f-b053b2004be0 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:35.617057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd34e3-3afa-4375-8b5f-b053b2004be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:35.618658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-46dd34e3-3afa-4375-8b5f-b053b2004be0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:35.621000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:35.720697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd34e3-3afa-4375-8b5f-b053b2004be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:35.722433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd34e3-3afa-4375-8b5f-b053b2004be0 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:35.743611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 704/1409] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:45.010536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=19 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:45.740535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=44 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:48.535927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27572da2-75ec-40ef-8786-9d00dd6ea129 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:48.536490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27572da2-75ec-40ef-8786-9d00dd6ea129 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:48.536490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27572da2-75ec-40ef-8786-9d00dd6ea129 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:48.538730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 706/1410] 10.222.0.50 () {54 vars in 918 bytes} [Thu Sep 15 10:49:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:48.544057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf876b-97b0-4627-8d90-8b4097b564cc None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:48.544549 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf876b-97b0-4627-8d90-8b4097b564cc None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:48.544654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf876b-97b0-4627-8d90-8b4097b564cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:48.621374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-bbbf876b-97b0-4627-8d90-8b4097b564cc None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:48.622604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-bbbf876b-97b0-4627-8d90-8b4097b564cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:48.728978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 705/1411] 10.222.0.50 () {58 vars in 1015 bytes} [Thu Sep 15 10:49:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:49:48.743710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002a8c1-4ecc-4393-ad0a-cb912e441cfb None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:48.744096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002a8c1-4ecc-4393-ad0a-cb912e441cfb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:48.744375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002a8c1-4ecc-4393-ad0a-cb912e441cfb None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:48.745736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 707/1412] 10.222.0.50 () {54 vars in 918 bytes} [Thu Sep 15 10:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:48.750450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a042e-2776-41a1-a34e-10b5ef02ecd9 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:48.750739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a042e-2776-41a1-a34e-10b5ef02ecd9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:48.751017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a042e-2776-41a1-a34e-10b5ef02ecd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:48.824227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:48.827334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-440a042e-2776-41a1-a34e-10b5ef02ecd9 None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:48.829111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-440a042e-2776-41a1-a34e-10b5ef02ecd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:48.944618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 706/1413] 10.222.0.50 () {58 vars in 1015 bytes} [Thu Sep 15 10:49:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:51.563692 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f51e1-13e0-43d9-969b-129ec4960a92 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:51.564116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f51e1-13e0-43d9-969b-129ec4960a92 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:51.564242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f51e1-13e0-43d9-969b-129ec4960a92 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:51.565863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:51.571914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5879641-7d86-4a13-9a32-b59635770e04 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:51.572369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5879641-7d86-4a13-9a32-b59635770e04 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:51.572486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5879641-7d86-4a13-9a32-b59635770e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:51.638419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-a5879641-7d86-4a13-9a32-b59635770e04 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:51.639633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a5879641-7d86-4a13-9a32-b59635770e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:51.695085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:51.737371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:52.029692 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f51e1-13e0-43d9-969b-129ec4960a92 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:52.031235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4c1f51e1-13e0-43d9-969b-129ec4960a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:52.176225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f51e1-13e0-43d9-969b-129ec4960a92 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:52.249750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:52.291414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fee2c-1ade-4ea5-8dbf-33264f42c2db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:52.291927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fee2c-1ade-4ea5-8dbf-33264f42c2db service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:52.292198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fee2c-1ade-4ea5-8dbf-33264f42c2db service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:52.292472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fee2c-1ade-4ea5-8dbf-33264f42c2db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:52.294064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-402fee2c-1ade-4ea5-8dbf-33264f42c2db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:52.382618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fee2c-1ade-4ea5-8dbf-33264f42c2db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:52.384227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fee2c-1ade-4ea5-8dbf-33264f42c2db service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:52.402575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 709/1416] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:53.540505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801570e6-bd44-4ddf-9857-b5b0c9b58ff8 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:53.541060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801570e6-bd44-4ddf-9857-b5b0c9b58ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:53.541169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801570e6-bd44-4ddf-9857-b5b0c9b58ff8 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:53.542449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:53.548922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc216a5-714a-4ef0-a64e-58fc25574bfe None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:53.549375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc216a5-714a-4ef0-a64e-58fc25574bfe None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:53.549474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc216a5-714a-4ef0-a64e-58fc25574bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:53.625041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0fc216a5-714a-4ef0-a64e-58fc25574bfe None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:53.626250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0fc216a5-714a-4ef0-a64e-58fc25574bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:53.653394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:53.723209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:53.874312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801570e6-bd44-4ddf-9857-b5b0c9b58ff8 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:53.875762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-801570e6-bd44-4ddf-9857-b5b0c9b58ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:54.023917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801570e6-bd44-4ddf-9857-b5b0c9b58ff8 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:54.065458 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:54.140101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95586be-6d9f-42bc-b7f5-41445fd5f3f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:54.140638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95586be-6d9f-42bc-b7f5-41445fd5f3f9 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:54.140946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95586be-6d9f-42bc-b7f5-41445fd5f3f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:54.141218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95586be-6d9f-42bc-b7f5-41445fd5f3f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:54.142798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a95586be-6d9f-42bc-b7f5-41445fd5f3f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:54.234251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95586be-6d9f-42bc-b7f5-41445fd5f3f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:54.235892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95586be-6d9f-42bc-b7f5-41445fd5f3f9 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:54.254497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 709/1419] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:55.382025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e620b-aab1-403d-9bba-28d896851e50 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:55.382610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e620b-aab1-403d-9bba-28d896851e50 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:55.382610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e620b-aab1-403d-9bba-28d896851e50 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:55.383926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:55.390038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1278c-7eb8-48ac-b16c-2ddd93aba195 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:55.390512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1278c-7eb8-48ac-b16c-2ddd93aba195 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:55.390614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1278c-7eb8-48ac-b16c-2ddd93aba195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:55.445269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:55.468470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-fce1278c-7eb8-48ac-b16c-2ddd93aba195 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:55.469701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fce1278c-7eb8-48ac-b16c-2ddd93aba195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:55.564091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:55.719923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e620b-aab1-403d-9bba-28d896851e50 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:55.721721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a56e620b-aab1-403d-9bba-28d896851e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:55.870702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e620b-aab1-403d-9bba-28d896851e50 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:55.882600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:55.981808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246aaac-3fc3-43e7-8b8e-8130406439c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:55.982338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246aaac-3fc3-43e7-8b8e-8130406439c1 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:55.982617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246aaac-3fc3-43e7-8b8e-8130406439c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:55.982891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246aaac-3fc3-43e7-8b8e-8130406439c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:55.984475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0246aaac-3fc3-43e7-8b8e-8130406439c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:56.073479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246aaac-3fc3-43e7-8b8e-8130406439c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:56.075070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246aaac-3fc3-43e7-8b8e-8130406439c1 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:56.093337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 712/1422] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:57.219918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb1ab-6d8b-4bcf-a4ff-db4212340bdc None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:57.220467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb1ab-6d8b-4bcf-a4ff-db4212340bdc None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:57.220467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb1ab-6d8b-4bcf-a4ff-db4212340bdc None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:57.221867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:57.228881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd275dd8-9cdd-4310-962f-84d7c018a732 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:57.229335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd275dd8-9cdd-4310-962f-84d7c018a732 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:57.229442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd275dd8-9cdd-4310-962f-84d7c018a732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:57.249364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:57.307394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dd275dd8-9cdd-4310-962f-84d7c018a732 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:57.308590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dd275dd8-9cdd-4310-962f-84d7c018a732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:57.404777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:57.555813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fb1ab-6d8b-4bcf-a4ff-db4212340bdc None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:57.557397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e44fb1ab-6d8b-4bcf-a4ff-db4212340bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:57.669884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:57.707073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fb1ab-6d8b-4bcf-a4ff-db4212340bdc None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:57.820905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d473de-347b-4a1f-83db-a1def8f94a35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:57.821460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d473de-347b-4a1f-83db-a1def8f94a35 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:57.821740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d473de-347b-4a1f-83db-a1def8f94a35 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:57.822012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d473de-347b-4a1f-83db-a1def8f94a35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:57.823682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-49d473de-347b-4a1f-83db-a1def8f94a35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:57.899704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:57.913592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d473de-347b-4a1f-83db-a1def8f94a35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:57.915196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d473de-347b-4a1f-83db-a1def8f94a35 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:57.933957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 712/1425] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:49:59.058148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5081a7-934d-47cb-b974-a1bda99ab744 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:59.058696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5081a7-934d-47cb-b974-a1bda99ab744 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:59.058696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5081a7-934d-47cb-b974-a1bda99ab744 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:59.060037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:49:59.066814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5bb56-38c7-4936-9795-d2d1045a84ad None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:59.067242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5bb56-38c7-4936-9795-d2d1045a84ad None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:59.067415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5bb56-38c7-4936-9795-d2d1045a84ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:59.144359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-61e5bb56-38c7-4936-9795-d2d1045a84ad None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:49:59.145599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-61e5bb56-38c7-4936-9795-d2d1045a84ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:59.246532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:49:59.400138 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5081a7-934d-47cb-b974-a1bda99ab744 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:49:59.401728 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6d5081a7-934d-47cb-b974-a1bda99ab744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:59.486921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:59.549339 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5081a7-934d-47cb-b974-a1bda99ab744 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:49:59.664581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9552c-cd40-44d7-99d5-aab756fc9be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:49:59.665139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9552c-cd40-44d7-99d5-aab756fc9be7 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:49:59.665417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9552c-cd40-44d7-99d5-aab756fc9be7 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:49:59.665693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9552c-cd40-44d7-99d5-aab756fc9be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:49:59.667285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3bf9552c-cd40-44d7-99d5-aab756fc9be7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:49:59.715683 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:49:59.757045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9552c-cd40-44d7-99d5-aab756fc9be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:49:59.758646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9552c-cd40-44d7-99d5-aab756fc9be7 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:49:59.776865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 715/1428] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:00.844117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed777483-3564-4195-bb9c-c30732896ef2 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:00.844757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed777483-3564-4195-bb9c-c30732896ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:00.844757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed777483-3564-4195-bb9c-c30732896ef2 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:00.846072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:00.853053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ca93b-06f5-4ba7-b431-69ed7b2a34dd None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:00.853533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ca93b-06f5-4ba7-b431-69ed7b2a34dd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:00.853646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ca93b-06f5-4ba7-b431-69ed7b2a34dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:00.930336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7a0ca93b-06f5-4ba7-b431-69ed7b2a34dd None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:00.931548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7a0ca93b-06f5-4ba7-b431-69ed7b2a34dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:01.026864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:01.178468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed777483-3564-4195-bb9c-c30732896ef2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:01.180006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ed777483-3564-4195-bb9c-c30732896ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:01.230233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:01.328092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed777483-3564-4195-bb9c-c30732896ef2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:01.439747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776b00d-7c97-43b0-ab44-4ebd36189dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:01.440349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776b00d-7c97-43b0-ab44-4ebd36189dc7 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:01.440918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776b00d-7c97-43b0-ab44-4ebd36189dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:01.441192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776b00d-7c97-43b0-ab44-4ebd36189dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:01.442776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a776b00d-7c97-43b0-ab44-4ebd36189dc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:01.450302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:01.534188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776b00d-7c97-43b0-ab44-4ebd36189dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:01.535806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776b00d-7c97-43b0-ab44-4ebd36189dc7 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:01.554585 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 715/1431] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:02.606709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf081c-6136-4111-baff-347756325e3d None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:02.607191 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf081c-6136-4111-baff-347756325e3d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:02.607296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf081c-6136-4111-baff-347756325e3d None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:02.608607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:02.614922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8436b-402c-4901-8429-beeeef1cf704 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:02.615369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8436b-402c-4901-8429-beeeef1cf704 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:02.615480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8436b-402c-4901-8429-beeeef1cf704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:02.692961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-40a8436b-402c-4901-8429-beeeef1cf704 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:02.694206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-40a8436b-402c-4901-8429-beeeef1cf704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:02.763459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:02.793581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:02.944346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf081c-6136-4111-baff-347756325e3d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:02.945898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-48cf081c-6136-4111-baff-347756325e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:02.964037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:03.094934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf081c-6136-4111-baff-347756325e3d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:03.182467 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:03.207877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52f243-01c8-44c8-a18a-99e987e53e64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:03.208497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f243-01c8-44c8-a18a-99e987e53e64 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:03.208780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f243-01c8-44c8-a18a-99e987e53e64 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:03.209088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f243-01c8-44c8-a18a-99e987e53e64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:03.211212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1d52f243-01c8-44c8-a18a-99e987e53e64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:03.300814 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f243-01c8-44c8-a18a-99e987e53e64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:03.302442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f243-01c8-44c8-a18a-99e987e53e64 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:03.320268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 718/1434] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:04.381072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4afa4-97b7-4acb-9603-5529859fbc2a None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:04.381882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4afa4-97b7-4acb-9603-5529859fbc2a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:04.381882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4afa4-97b7-4acb-9603-5529859fbc2a None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:04.383193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:04.389497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26aa03-21d4-453e-8ca5-87723f7204cb None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:04.389932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26aa03-21d4-453e-8ca5-87723f7204cb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:04.390052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26aa03-21d4-453e-8ca5-87723f7204cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:04.466120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ee26aa03-21d4-453e-8ca5-87723f7204cb None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:04.467337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ee26aa03-21d4-453e-8ca5-87723f7204cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:04.509629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:04.565083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:04.717633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4afa4-97b7-4acb-9603-5529859fbc2a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:04.719208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-acc4afa4-97b7-4acb-9603-5529859fbc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:04.864006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4afa4-97b7-4acb-9603-5529859fbc2a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:04.920010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:04.981517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab69e2-9181-4d4b-a940-f0c13a8214fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:04.982045 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab69e2-9181-4d4b-a940-f0c13a8214fe service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:04.982322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab69e2-9181-4d4b-a940-f0c13a8214fe service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:04.982590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab69e2-9181-4d4b-a940-f0c13a8214fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:04.984168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-10ab69e2-9181-4d4b-a940-f0c13a8214fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:05.073907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab69e2-9181-4d4b-a940-f0c13a8214fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:05.075525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab69e2-9181-4d4b-a940-f0c13a8214fe service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:05.093686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 718/1437] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:06.155410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a879054-291b-4ff2-856c-b2b3fc3982b1 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:06.156001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a879054-291b-4ff2-856c-b2b3fc3982b1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:06.156001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a879054-291b-4ff2-856c-b2b3fc3982b1 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:06.157367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:06.165624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd716c9-75ad-4576-b2d0-d73f926492eb None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:06.166177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd716c9-75ad-4576-b2d0-d73f926492eb None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:06.166177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd716c9-75ad-4576-b2d0-d73f926492eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:06.252155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-fbd716c9-75ad-4576-b2d0-d73f926492eb None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:06.253379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fbd716c9-75ad-4576-b2d0-d73f926492eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:06.255389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:06.352468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:06.504030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a879054-291b-4ff2-856c-b2b3fc3982b1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:06.505557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1a879054-291b-4ff2-856c-b2b3fc3982b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:06.650781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a879054-291b-4ff2-856c-b2b3fc3982b1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:06.677247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:06.763787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214719c-4685-4ebb-8b33-de914efc2855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:06.764317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214719c-4685-4ebb-8b33-de914efc2855 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:06.764600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214719c-4685-4ebb-8b33-de914efc2855 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:06.764904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214719c-4685-4ebb-8b33-de914efc2855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:06.766502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7214719c-4685-4ebb-8b33-de914efc2855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:06.856255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214719c-4685-4ebb-8b33-de914efc2855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:06.857895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214719c-4685-4ebb-8b33-de914efc2855 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:06.875498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 721/1440] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:08.975879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4df8cb-b79b-496b-b857-c9adf5421650 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:08.976453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4df8cb-b79b-496b-b857-c9adf5421650 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:08.976453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4df8cb-b79b-496b-b857-c9adf5421650 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:08.977818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:50:08.983607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37edf0-8bdc-4ba9-8b93-5a660751e323 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:08.984028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37edf0-8bdc-4ba9-8b93-5a660751e323 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:08.984224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37edf0-8bdc-4ba9-8b93-5a660751e323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:09.027057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:09.051650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1f37edf0-8bdc-4ba9-8b93-5a660751e323 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:09.052900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1f37edf0-8bdc-4ba9-8b93-5a660751e323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:09.149745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:09.440429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4df8cb-b79b-496b-b857-c9adf5421650 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:09.442269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2f4df8cb-b79b-496b-b857-c9adf5421650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:09.548255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4df8cb-b79b-496b-b857-c9adf5421650 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:09.585806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:09.629104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f228c-d9c6-4070-aa78-2ba7910a44a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:09.629719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f228c-d9c6-4070-aa78-2ba7910a44a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:09.629981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f228c-d9c6-4070-aa78-2ba7910a44a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:09.630248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f228c-d9c6-4070-aa78-2ba7910a44a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:09.631845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b09f228c-d9c6-4070-aa78-2ba7910a44a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:09.742008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f228c-d9c6-4070-aa78-2ba7910a44a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:09.743381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f228c-d9c6-4070-aa78-2ba7910a44a3 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:09.761331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 721/1443] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:12.065119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80394207-c52d-4708-a739-dbbf0590d98d None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:12.065710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80394207-c52d-4708-a739-dbbf0590d98d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:12.065710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80394207-c52d-4708-a739-dbbf0590d98d None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:12.067266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:12.073123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cbcf2-c9ba-433d-95c7-94511fce6c84 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:12.073538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cbcf2-c9ba-433d-95c7-94511fce6c84 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:12.073679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cbcf2-c9ba-433d-95c7-94511fce6c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:12.139748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ca6cbcf2-c9ba-433d-95c7-94511fce6c84 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:12.140990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ca6cbcf2-c9ba-433d-95c7-94511fce6c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:12.154415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:12.240159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:12.526978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80394207-c52d-4708-a739-dbbf0590d98d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:12.528535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-80394207-c52d-4708-a739-dbbf0590d98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:12.633373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80394207-c52d-4708-a739-dbbf0590d98d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:12.646143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:12.708163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbe7f6-5726-460a-96e3-0a460edb0864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:12.708767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbe7f6-5726-460a-96e3-0a460edb0864 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:12.709070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbe7f6-5726-460a-96e3-0a460edb0864 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:12.709337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbe7f6-5726-460a-96e3-0a460edb0864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:12.711886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bebbe7f6-5726-460a-96e3-0a460edb0864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:12.820002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbe7f6-5726-460a-96e3-0a460edb0864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:12.821426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbe7f6-5726-460a-96e3-0a460edb0864 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:12.839673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 724/1446] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:15.318200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbb54e-7fe0-467a-94db-2323cbc5a971 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:15.318740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbb54e-7fe0-467a-94db-2323cbc5a971 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:15.318740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbb54e-7fe0-467a-94db-2323cbc5a971 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:15.320936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:15.326151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7916cda-dca7-4bf4-a3ee-6b4b35d566d9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:15.326612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7916cda-dca7-4bf4-a3ee-6b4b35d566d9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:15.326710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7916cda-dca7-4bf4-a3ee-6b4b35d566d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:15.379889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:15.399015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f7916cda-dca7-4bf4-a3ee-6b4b35d566d9 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:15.400217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f7916cda-dca7-4bf4-a3ee-6b4b35d566d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:15.496863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:15.782110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbb54e-7fe0-467a-94db-2323cbc5a971 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:15.783630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f5fbb54e-7fe0-467a-94db-2323cbc5a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:15.886737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbb54e-7fe0-467a-94db-2323cbc5a971 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:15.926270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:15.961198 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f81ad4-db65-4f46-ba0e-a0b8608d5bee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:15.961802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f81ad4-db65-4f46-ba0e-a0b8608d5bee service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:15.962081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f81ad4-db65-4f46-ba0e-a0b8608d5bee service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:15.962351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f81ad4-db65-4f46-ba0e-a0b8608d5bee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:15.963926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f9f81ad4-db65-4f46-ba0e-a0b8608d5bee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:16.076307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f81ad4-db65-4f46-ba0e-a0b8608d5bee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:16.077689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f81ad4-db65-4f46-ba0e-a0b8608d5bee service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:16.095241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 724/1449] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:18.386262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa18d9-c91b-4966-9fd0-6255e1d84171 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:18.386843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa18d9-c91b-4966-9fd0-6255e1d84171 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:18.386843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa18d9-c91b-4966-9fd0-6255e1d84171 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:18.388182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:18.393432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc830ff-9f2c-4b10-82de-75f821959926 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:18.393832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc830ff-9f2c-4b10-82de-75f821959926 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:18.393999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc830ff-9f2c-4b10-82de-75f821959926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:18.461809 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7dc830ff-9f2c-4b10-82de-75f821959926 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:18.463023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7dc830ff-9f2c-4b10-82de-75f821959926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:18.484250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:18.563330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:18.850390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa18d9-c91b-4966-9fd0-6255e1d84171 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:18.852547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f3fa18d9-c91b-4966-9fd0-6255e1d84171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:18.954745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa18d9-c91b-4966-9fd0-6255e1d84171 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:18.975271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:19.031424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c66f85-7525-4575-8f76-1df4343245c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:19.032032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c66f85-7525-4575-8f76-1df4343245c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:19.032303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c66f85-7525-4575-8f76-1df4343245c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:19.032568 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c66f85-7525-4575-8f76-1df4343245c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:19.034172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-19c66f85-7525-4575-8f76-1df4343245c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:19.141630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c66f85-7525-4575-8f76-1df4343245c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:19.143006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c66f85-7525-4575-8f76-1df4343245c5 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:19.161180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 727/1452] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:21.376277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc88d8-516b-47d2-ab74-f1c144477eda None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:21.376849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc88d8-516b-47d2-ab74-f1c144477eda None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:21.376849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc88d8-516b-47d2-ab74-f1c144477eda None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:21.378218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:21.384446 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1ea2e-3904-4c28-b11b-b5758c84d12a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:21.384868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1ea2e-3904-4c28-b11b-b5758c84d12a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:21.385032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1ea2e-3904-4c28-b11b-b5758c84d12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:21.448532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:21.451325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cda1ea2e-3904-4c28-b11b-b5758c84d12a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:21.453162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cda1ea2e-3904-4c28-b11b-b5758c84d12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:21.550697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:21.837949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc88d8-516b-47d2-ab74-f1c144477eda None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:21.839526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5ddc88d8-516b-47d2-ab74-f1c144477eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:21.941903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc88d8-516b-47d2-ab74-f1c144477eda None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:21.990418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:22.016025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4813a7e-587a-4d23-ba36-8cc596998ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:22.016633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4813a7e-587a-4d23-ba36-8cc596998ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:22.016943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4813a7e-587a-4d23-ba36-8cc596998ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:22.017224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4813a7e-587a-4d23-ba36-8cc596998ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:22.018815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b4813a7e-587a-4d23-ba36-8cc596998ae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:22.127292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4813a7e-587a-4d23-ba36-8cc596998ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:22.128656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4813a7e-587a-4d23-ba36-8cc596998ae9 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:22.147042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 727/1455] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:24.368122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08eeed-0f44-4850-90a8-aea468bd1734 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:24.368664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08eeed-0f44-4850-90a8-aea468bd1734 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:24.368764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08eeed-0f44-4850-90a8-aea468bd1734 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:24.370129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:24.375384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74b7a0-fb17-49e9-b81d-d795a45c84fa None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:24.375837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74b7a0-fb17-49e9-b81d-d795a45c84fa None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:24.375946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74b7a0-fb17-49e9-b81d-d795a45c84fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:24.442932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ba74b7a0-fb17-49e9-b81d-d795a45c84fa None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:24.444127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ba74b7a0-fb17-49e9-b81d-d795a45c84fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:24.473725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:24.542133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:24.829508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08eeed-0f44-4850-90a8-aea468bd1734 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:24.831071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8f08eeed-0f44-4850-90a8-aea468bd1734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:24.936886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08eeed-0f44-4850-90a8-aea468bd1734 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:24.965789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:25.014230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e7d52-b603-4fc5-902e-5257a734f512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:25.014878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e7d52-b603-4fc5-902e-5257a734f512 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:25.015164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e7d52-b603-4fc5-902e-5257a734f512 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:25.015441 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e7d52-b603-4fc5-902e-5257a734f512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:25.017092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-312e7d52-b603-4fc5-902e-5257a734f512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:25.124687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e7d52-b603-4fc5-902e-5257a734f512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:25.126083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e7d52-b603-4fc5-902e-5257a734f512 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:25.144315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 730/1458] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:27.381102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa81e0-30c8-4eeb-8143-7004fa67afed None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:27.381616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa81e0-30c8-4eeb-8143-7004fa67afed None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:27.381616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa81e0-30c8-4eeb-8143-7004fa67afed None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:27.382982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:27.388269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad451b93-965f-42f1-86e7-831b8fdf0be2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:27.388702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad451b93-965f-42f1-86e7-831b8fdf0be2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:27.388823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad451b93-965f-42f1-86e7-831b8fdf0be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:27.454695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ad451b93-965f-42f1-86e7-831b8fdf0be2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:27.455888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ad451b93-965f-42f1-86e7-831b8fdf0be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:27.464820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:27.555340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:27.841640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa81e0-30c8-4eeb-8143-7004fa67afed None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:27.843178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-16fa81e0-30c8-4eeb-8143-7004fa67afed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:27.950696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa81e0-30c8-4eeb-8143-7004fa67afed None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:28.125668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:28.143215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70728e6-f2b2-4860-8107-91b2fd9ca21f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:28.143816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70728e6-f2b2-4860-8107-91b2fd9ca21f service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:28.144090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70728e6-f2b2-4860-8107-91b2fd9ca21f service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:28.144358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70728e6-f2b2-4860-8107-91b2fd9ca21f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:28.145984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e70728e6-f2b2-4860-8107-91b2fd9ca21f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:28.254662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70728e6-f2b2-4860-8107-91b2fd9ca21f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:28.256095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70728e6-f2b2-4860-8107-91b2fd9ca21f service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:28.274354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 730/1461] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:30.500494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0deabc-50b9-4620-b041-cbde158e9a4a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:30.501127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0deabc-50b9-4620-b041-cbde158e9a4a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:30.501127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0deabc-50b9-4620-b041-cbde158e9a4a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:30.502463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:30.508467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60125811-bb18-4579-a21f-cfb2a178e26e None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:30.508968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60125811-bb18-4579-a21f-cfb2a178e26e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:30.509062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60125811-bb18-4579-a21f-cfb2a178e26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:30.577423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-60125811-bb18-4579-a21f-cfb2a178e26e None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:30.578625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-60125811-bb18-4579-a21f-cfb2a178e26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:30.616430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:30.675610 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:30.977074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0deabc-50b9-4620-b041-cbde158e9a4a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:30.978645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dd0deabc-50b9-4620-b041-cbde158e9a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:31.086184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0deabc-50b9-4620-b041-cbde158e9a4a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:31.123781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:31.159159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e88b4-ed55-401d-8033-f7259ad7ca2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:31.159768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e88b4-ed55-401d-8033-f7259ad7ca2d service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:31.160050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e88b4-ed55-401d-8033-f7259ad7ca2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:31.160342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e88b4-ed55-401d-8033-f7259ad7ca2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:31.161956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e89e88b4-ed55-401d-8033-f7259ad7ca2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:31.271458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e88b4-ed55-401d-8033-f7259ad7ca2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:31.272973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e88b4-ed55-401d-8033-f7259ad7ca2d service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:31.290861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 733/1464] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:33.486278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9559ce-8452-43e8-bd3c-0babf93ef970 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:33.486869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9559ce-8452-43e8-bd3c-0babf93ef970 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:33.486869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9559ce-8452-43e8-bd3c-0babf93ef970 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:33.488188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:33.493845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b549c-dccc-40a2-98f6-01c23e8d3d5f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:33.494267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b549c-dccc-40a2-98f6-01c23e8d3d5f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:33.494421 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b549c-dccc-40a2-98f6-01c23e8d3d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:33.561758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-685b549c-dccc-40a2-98f6-01c23e8d3d5f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:33.562974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-685b549c-dccc-40a2-98f6-01c23e8d3d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:33.581749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:33.660674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:33.947173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9559ce-8452-43e8-bd3c-0babf93ef970 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:33.948696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6d9559ce-8452-43e8-bd3c-0babf93ef970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:34.053546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9559ce-8452-43e8-bd3c-0babf93ef970 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:34.119350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:34.126650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48074055-5d67-4529-a82b-57aaddfd88df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:34.127257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48074055-5d67-4529-a82b-57aaddfd88df service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:34.127533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48074055-5d67-4529-a82b-57aaddfd88df service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:34.127806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48074055-5d67-4529-a82b-57aaddfd88df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:34.129587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-48074055-5d67-4529-a82b-57aaddfd88df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:34.237548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48074055-5d67-4529-a82b-57aaddfd88df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:34.238935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48074055-5d67-4529-a82b-57aaddfd88df service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:34.258211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 733/1467] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:36.598316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec286272-bbd1-4717-9755-8d0244c13672 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:36.598857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec286272-bbd1-4717-9755-8d0244c13672 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:36.598857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec286272-bbd1-4717-9755-8d0244c13672 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:36.600206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:36.605835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66724288-f668-4877-be79-bccdf3fc3132 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:36.606287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66724288-f668-4877-be79-bccdf3fc3132 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:36.606393 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66724288-f668-4877-be79-bccdf3fc3132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:36.672688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-66724288-f668-4877-be79-bccdf3fc3132 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:36.673935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-66724288-f668-4877-be79-bccdf3fc3132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:36.722441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:36.770477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:37.056795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec286272-bbd1-4717-9755-8d0244c13672 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:37.058408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ec286272-bbd1-4717-9755-8d0244c13672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:37.160845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec286272-bbd1-4717-9755-8d0244c13672 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:37.326041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:37.352999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13afa0b-997f-4000-b6bf-854b25cc8dc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:37.353608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13afa0b-997f-4000-b6bf-854b25cc8dc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:37.353899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13afa0b-997f-4000-b6bf-854b25cc8dc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:37.354146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13afa0b-997f-4000-b6bf-854b25cc8dc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:37.355704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e13afa0b-997f-4000-b6bf-854b25cc8dc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:37.470394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13afa0b-997f-4000-b6bf-854b25cc8dc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:37.471839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13afa0b-997f-4000-b6bf-854b25cc8dc2 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:37.489409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 736/1470] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:50:46.769001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=22 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:47.487148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=66 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:57.529638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fe7f7-92fc-4443-9d16-ba65686b265f None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:57.530239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fe7f7-92fc-4443-9d16-ba65686b265f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:57.530239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fe7f7-92fc-4443-9d16-ba65686b265f None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:57.531537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 735/1471] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:57.536792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f5f9e-6ecd-416a-b3a6-e886545de505 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:57.537234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f5f9e-6ecd-416a-b3a6-e886545de505 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:57.537361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f5f9e-6ecd-416a-b3a6-e886545de505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:57.602915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c84f5f9e-6ecd-416a-b3a6-e886545de505 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:57.604043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c84f5f9e-6ecd-416a-b3a6-e886545de505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:57.699142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 737/1472] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:58.004965 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cbb42-8b97-41cf-b3c5-45b957ad6d09 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:58.005251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cbb42-8b97-41cf-b3c5-45b957ad6d09 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:58.005523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cbb42-8b97-41cf-b3c5-45b957ad6d09 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:58.006884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 736/1473] 10.222.0.50 () {54 vars in 962 bytes} [Thu Sep 15 10:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:50:58.012561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebf4c2-eca8-422d-9fdf-cdfec3f8c617 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:58.012885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebf4c2-eca8-422d-9fdf-cdfec3f8c617 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:58.013167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebf4c2-eca8-422d-9fdf-cdfec3f8c617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:58.090496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d6ebf4c2-eca8-422d-9fdf-cdfec3f8c617 None None] MFA Rules not processed for user `6908a326a3d3437c9db898976a2def1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:50:58.091707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d6ebf4c2-eca8-422d-9fdf-cdfec3f8c617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:58.111766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:58.170352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 738/1474] 10.222.0.50 () {58 vars in 1059 bytes} [Thu Sep 15 10:50:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:50:58.178098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cbb42-8b97-41cf-b3c5-45b957ad6d09 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:50:58.180247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-502cbb42-8b97-41cf-b3c5-45b957ad6d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:58.282799 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cbb42-8b97-41cf-b3c5-45b957ad6d09 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:50:58.356849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581d84e-ccbd-45ce-b598-8cb9b86c9076 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:50:58.357454 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581d84e-ccbd-45ce-b598-8cb9b86c9076 service glance] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:50:58.357729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581d84e-ccbd-45ce-b598-8cb9b86c9076 service glance] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:50:58.357993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581d84e-ccbd-45ce-b598-8cb9b86c9076 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:50:58.359572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0581d84e-ccbd-45ce-b598-8cb9b86c9076 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:50:58.414519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:50:58.469814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581d84e-ccbd-45ce-b598-8cb9b86c9076 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:50:58.471267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581d84e-ccbd-45ce-b598-8cb9b86c9076 service glance] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:50:58.490111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 737/1475] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:51:00.749708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7a83f-1337-4d7c-9bb1-6ff458d6c49e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:00.750240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7a83f-1337-4d7c-9bb1-6ff458d6c49e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:00.750240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7a83f-1337-4d7c-9bb1-6ff458d6c49e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:00.751593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 739/1476] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:51:00.757035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b67a2a-09ae-4202-ae09-49f57b4c2339 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:00.757482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b67a2a-09ae-4202-ae09-49f57b4c2339 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:00.757613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b67a2a-09ae-4202-ae09-49f57b4c2339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:00.826423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d7b67a2a-09ae-4202-ae09-49f57b4c2339 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:51:00.827718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d7b67a2a-09ae-4202-ae09-49f57b4c2339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:00.924702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 738/1477] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:51:01.200041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7a83f-1337-4d7c-9bb1-6ff458d6c49e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:51:01.201712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-23c7a83f-1337-4d7c-9bb1-6ff458d6c49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:01.327540 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7a83f-1337-4d7c-9bb1-6ff458d6c49e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:51:01.376297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:01.422794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7850a2-3e20-49d7-9e85-536590c4a007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:51:01.423402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7850a2-3e20-49d7-9e85-536590c4a007 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:01.423685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7850a2-3e20-49d7-9e85-536590c4a007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:01.423953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7850a2-3e20-49d7-9e85-536590c4a007 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:01.424583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7850a2-3e20-49d7-9e85-536590c4a007 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:51:01.426360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-0d7850a2-3e20-49d7-9e85-536590c4a007 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:51:01.426360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 15 10:51:01.426360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: Sep 15 10:51:01.427502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7850a2-3e20-49d7-9e85-536590c4a007 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:51:01.447141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 740/1478] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Sep 15 10:51:01 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:51:03.608632 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03773d13-4a09-4d0c-a138-09ce0b478292 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:03.609171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03773d13-4a09-4d0c-a138-09ce0b478292 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:03.609270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03773d13-4a09-4d0c-a138-09ce0b478292 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:03.610545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 739/1479] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:51:03.615727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32a3c-1203-41f2-b5ea-c49d1d82d827 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:03.616170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32a3c-1203-41f2-b5ea-c49d1d82d827 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:03.616340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32a3c-1203-41f2-b5ea-c49d1d82d827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:03.682586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dce32a3c-1203-41f2-b5ea-c49d1d82d827 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:51:03.683796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dce32a3c-1203-41f2-b5ea-c49d1d82d827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:03.780079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 741/1480] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:51:04.075484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03773d13-4a09-4d0c-a138-09ce0b478292 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:51:04.077449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-03773d13-4a09-4d0c-a138-09ce0b478292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:04.087419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:04.223419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03773d13-4a09-4d0c-a138-09ce0b478292 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:51:04.303387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:04.337068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a121c-4f75-4c5c-b425-de917047ecbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:51:04.337603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a121c-4f75-4c5c-b425-de917047ecbd service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:04.337880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a121c-4f75-4c5c-b425-de917047ecbd service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:04.338150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a121c-4f75-4c5c-b425-de917047ecbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:04.339740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9e3a121c-4f75-4c5c-b425-de917047ecbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:04.427769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a121c-4f75-4c5c-b425-de917047ecbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:51:04.429393 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a121c-4f75-4c5c-b425-de917047ecbd service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:51:04.447805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 740/1481] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:51:06.783146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe17ded-5a00-4a7f-b41c-46e1f5026b30 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:06.783675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe17ded-5a00-4a7f-b41c-46e1f5026b30 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:06.783675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe17ded-5a00-4a7f-b41c-46e1f5026b30 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:06.784990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 742/1482] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:51:06.791156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a92a6-9b04-4632-b4e7-bdcd2c5a23ef None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:06.791593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a92a6-9b04-4632-b4e7-bdcd2c5a23ef None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:06.791725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a92a6-9b04-4632-b4e7-bdcd2c5a23ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:06.858030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4a3a92a6-9b04-4632-b4e7-bdcd2c5a23ef None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:51:06.859259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4a3a92a6-9b04-4632-b4e7-bdcd2c5a23ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:06.905163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:06.955748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 741/1483] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:51:07.246166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe17ded-5a00-4a7f-b41c-46e1f5026b30 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:51:07.247687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bfe17ded-5a00-4a7f-b41c-46e1f5026b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:07.272290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:07.353222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe17ded-5a00-4a7f-b41c-46e1f5026b30 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:51:07.425297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737acee-e653-4570-a89d-2b2cb295a11d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:51:07.425891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737acee-e653-4570-a89d-2b2cb295a11d service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:07.426167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737acee-e653-4570-a89d-2b2cb295a11d service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:07.426434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737acee-e653-4570-a89d-2b2cb295a11d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:07.427985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e737acee-e653-4570-a89d-2b2cb295a11d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:07.501561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:07.544888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737acee-e653-4570-a89d-2b2cb295a11d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:51:07.546330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737acee-e653-4570-a89d-2b2cb295a11d service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:51:07.564997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 743/1484] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:51:09.785944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69631d1a-1343-4288-bfed-0c79ef792f04 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:09.786516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69631d1a-1343-4288-bfed-0c79ef792f04 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:09.786516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69631d1a-1343-4288-bfed-0c79ef792f04 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:09.787858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 742/1485] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:51:09.793440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396f6a4-6287-45bb-be46-10129259134f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:09.793855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396f6a4-6287-45bb-be46-10129259134f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:09.793991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396f6a4-6287-45bb-be46-10129259134f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:09.860204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3396f6a4-6287-45bb-be46-10129259134f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:51:09.861382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3396f6a4-6287-45bb-be46-10129259134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:09.958487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 744/1486] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:51:10.249724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69631d1a-1343-4288-bfed-0c79ef792f04 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:51:10.251256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-69631d1a-1343-4288-bfed-0c79ef792f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:10.354491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69631d1a-1343-4288-bfed-0c79ef792f04 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:51:10.426235 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:10.427980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3aca2-ea31-4642-b0bf-4f04e01b0e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:51:10.428590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3aca2-ea31-4642-b0bf-4f04e01b0e45 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:10.429097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3aca2-ea31-4642-b0bf-4f04e01b0e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:10.429542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3aca2-ea31-4642-b0bf-4f04e01b0e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:10.431447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c0a3aca2-ea31-4642-b0bf-4f04e01b0e45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:10.545450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3aca2-ea31-4642-b0bf-4f04e01b0e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:51:10.546895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3aca2-ea31-4642-b0bf-4f04e01b0e45 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:51:10.564961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 743/1487] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:51:12.859302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1394fe5-5b9b-4351-894c-f1271ba6b811 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:12.859868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1394fe5-5b9b-4351-894c-f1271ba6b811 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:12.859868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1394fe5-5b9b-4351-894c-f1271ba6b811 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:12.861122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 745/1488] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:51:12.866323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02173aa3-d948-4719-9b53-3a295caef86d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:12.866741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02173aa3-d948-4719-9b53-3a295caef86d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:12.866885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02173aa3-d948-4719-9b53-3a295caef86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:12.933916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-02173aa3-d948-4719-9b53-3a295caef86d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:51:12.935113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-02173aa3-d948-4719-9b53-3a295caef86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:12.987977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:13.031948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 744/1489] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:51:13.324421 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1394fe5-5b9b-4351-894c-f1271ba6b811 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:51:13.326038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c1394fe5-5b9b-4351-894c-f1271ba6b811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:13.362175 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:13.430011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1394fe5-5b9b-4351-894c-f1271ba6b811 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:51:13.502055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa7ea9-87f3-4977-b2b7-0fa5e7c66559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:51:13.502652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa7ea9-87f3-4977-b2b7-0fa5e7c66559 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:13.502927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa7ea9-87f3-4977-b2b7-0fa5e7c66559 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:13.503195 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa7ea9-87f3-4977-b2b7-0fa5e7c66559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:13.504754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-33fa7ea9-87f3-4977-b2b7-0fa5e7c66559 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:13.585269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:13.615516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa7ea9-87f3-4977-b2b7-0fa5e7c66559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:51:13.616989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa7ea9-87f3-4977-b2b7-0fa5e7c66559 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:51:13.635656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 746/1490] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:51:16.005579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980096-ba9f-4e2f-a019-84d8a745207e None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:16.006166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980096-ba9f-4e2f-a019-84d8a745207e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:16.006166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980096-ba9f-4e2f-a019-84d8a745207e None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:16.007475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 745/1491] 10.222.0.50 () {54 vars in 927 bytes} [Thu Sep 15 10:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:51:16.012914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fe46b-7a1e-48e5-8e10-b8cd70df975f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:16.013373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fe46b-7a1e-48e5-8e10-b8cd70df975f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:16.013487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fe46b-7a1e-48e5-8e10-b8cd70df975f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:16.080567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5d3fe46b-7a1e-48e5-8e10-b8cd70df975f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:51:16.081821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5d3fe46b-7a1e-48e5-8e10-b8cd70df975f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:16.177699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 747/1492] 10.222.0.50 () {58 vars in 1024 bytes} [Thu Sep 15 10:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:51:16.472095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4980096-ba9f-4e2f-a019-84d8a745207e None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:51:16.473651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b4980096-ba9f-4e2f-a019-84d8a745207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:16.578886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4980096-ba9f-4e2f-a019-84d8a745207e None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:51:16.655320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a703d6-7125-44d2-945b-071a95c7d4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:51:16.655928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a703d6-7125-44d2-945b-071a95c7d4cb service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:51:16.656206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a703d6-7125-44d2-945b-071a95c7d4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:51:16.656477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a703d6-7125-44d2-945b-071a95c7d4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:51:16.658163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-87a703d6-7125-44d2-945b-071a95c7d4cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:51:16.667301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:16.768105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a703d6-7125-44d2-945b-071a95c7d4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:51:16.769505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a703d6-7125-44d2-945b-071a95c7d4cb service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:51:16.787570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 746/1493] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:51:26.176121 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=82 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:51:26.785360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=50 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:37.155911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b653952-a38e-4d10-85a2-09b51d2118c0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:37.156530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b653952-a38e-4d10-85a2-09b51d2118c0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:37.156530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b653952-a38e-4d10-85a2-09b51d2118c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:37.261468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5b653952-a38e-4d10-85a2-09b51d2118c0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:52:37.263931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5b653952-a38e-4d10-85a2-09b51d2118c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:37.384070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 748/1494] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:52:37.393192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bad72-f746-4914-9c54-6616038c98e9 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:37.393734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bad72-f746-4914-9c54-6616038c98e9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:37.393734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bad72-f746-4914-9c54-6616038c98e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:37.489190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-321bad72-f746-4914-9c54-6616038c98e9 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:52:37.490332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-321bad72-f746-4914-9c54-6616038c98e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:37.612093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 747/1495] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:52:37.616951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b653952-a38e-4d10-85a2-09b51d2118c0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:37.618200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5b653952-a38e-4d10-85a2-09b51d2118c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:37.629780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b653952-a38e-4d10-85a2-09b51d2118c0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:37.705867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:37.740993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:37.741516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:37.741790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:37.742056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:37.742693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:52:37.742991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:37.744003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:52:37.744003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 15 10:52:37.744003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: Sep 15 10:52:37.744926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:37.755894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 749/1496] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:52:37.764327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bad72-f746-4914-9c54-6616038c98e9 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:37.773382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bad72-f746-4914-9c54-6616038c98e9 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:37.834350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:37.869204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:37.869796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:37.870074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:37.870343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:37.870977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:37.872975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WARNING py.warnings [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:52:37.872975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: warnings.warn(msg) Sep 15 10:52:37.872975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: Sep 15 10:52:37.874185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:37.931926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 748/1497] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:52:37 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:52:37.936155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:37.937703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:37.972596 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82f079-6e37-486c-8f83-d69f66b60007 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:38.066725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:38.067248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:38.067528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:38.067792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:38.068414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:38.069455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:52:38.069455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 15 10:52:38.069455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: Sep 15 10:52:38.070660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:38.113126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 750/1498] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:52:37 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:52:38.117434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:38.119043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:38.187790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764fb7a-c3a6-4463-a06e-0daac85aa322 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:38.271304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:38.283064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:38.283664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:38.283937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:38.284201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:38.284858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:38.285858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WARNING py.warnings [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:52:38.285858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: warnings.warn(msg) Sep 15 10:52:38.285858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: Sep 15 10:52:38.286745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:38.299582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 749/1499] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:52:38.303820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:38.305401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:38.317442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0633b0-8f71-476b-b683-fa59c3366f0d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:38.342650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:38.413912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6107b1-cfd5-4328-851c-144544d174dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:38.414533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6107b1-cfd5-4328-851c-144544d174dc admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:38.414815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6107b1-cfd5-4328-851c-144544d174dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:38.415089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6107b1-cfd5-4328-851c-144544d174dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:38.415733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6107b1-cfd5-4328-851c-144544d174dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:38.416752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-cf6107b1-cfd5-4328-851c-144544d174dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:52:38.416752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 15 10:52:38.416752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: Sep 15 10:52:38.417673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6107b1-cfd5-4328-851c-144544d174dc admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:38.430442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 751/1500] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:52:38.434928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:38.436451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:38.470748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405c3c2-c64e-4e1d-820f-55d654ec9779 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:38.565232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42862af9-5b6e-41f2-845e-d9d84a116524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:38.565895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42862af9-5b6e-41f2-845e-d9d84a116524 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:38.566176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42862af9-5b6e-41f2-845e-d9d84a116524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:38.566446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42862af9-5b6e-41f2-845e-d9d84a116524 admin admin] PATH_INFO: `/v3/projects/23aec0466e1f42d7a22c7e72c15723f4/users/9c45a0f4d90146a98f1f9ede3107ae54/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:38.590050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42862af9-5b6e-41f2-845e-d9d84a116524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23aec0466e1f42d7a22c7e72c15723f4, user_id=9c45a0f4d90146a98f1f9ede3107ae54, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:38.591642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WARNING py.warnings [None req-42862af9-5b6e-41f2-845e-d9d84a116524 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:52:38.591642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: warnings.warn(msg) Sep 15 10:52:38.591642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: Sep 15 10:52:38.593668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42862af9-5b6e-41f2-845e-d9d84a116524 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:38.618659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 750/1501] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:52:38 2022] PUT /identity/v3/projects/23aec0466e1f42d7a22c7e72c15723f4/users/9c45a0f4d90146a98f1f9ede3107ae54/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:52:38.630926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094d70a-b849-4e05-b9e5-a826f7b2d8a3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:38.631213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094d70a-b849-4e05-b9e5-a826f7b2d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:38.631486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094d70a-b849-4e05-b9e5-a826f7b2d8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:38.689440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3094d70a-b849-4e05-b9e5-a826f7b2d8a3 None None] MFA Rules not processed for user `9c45a0f4d90146a98f1f9ede3107ae54`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:52:38.690643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3094d70a-b849-4e05-b9e5-a826f7b2d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:38.767236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 752/1502] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:52:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:52:38.781153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821fa8b-80d7-4a49-aa45-7c06adcaa712 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:38.781442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821fa8b-80d7-4a49-aa45-7c06adcaa712 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:38.781716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821fa8b-80d7-4a49-aa45-7c06adcaa712 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:38.783103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 751/1503] 10.222.0.50 () {54 vars in 964 bytes} [Thu Sep 15 10:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:52:38.787705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c79a1-fefc-4baf-9ca3-b671378a9357 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:38.787990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c79a1-fefc-4baf-9ca3-b671378a9357 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:38.788262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c79a1-fefc-4baf-9ca3-b671378a9357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:38.819576 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:38.865095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f23c79a1-fefc-4baf-9ca3-b671378a9357 None None] MFA Rules not processed for user `d7fa5d550a2d442a820cf4e12ebc36c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:52:38.867220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f23c79a1-fefc-4baf-9ca3-b671378a9357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:38.940936 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 753/1504] 10.222.0.50 () {58 vars in 1061 bytes} [Thu Sep 15 10:52:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:52:38.949089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821fa8b-80d7-4a49-aa45-7c06adcaa712 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:38.951151 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6821fa8b-80d7-4a49-aa45-7c06adcaa712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:39.001893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:39.058406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821fa8b-80d7-4a49-aa45-7c06adcaa712 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:39.134146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:39.134680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:39.134955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:39.135221 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:39.136843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:39.234099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:39.235538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:39.239007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:39.254742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 752/1505] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:52:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:52:39.299839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b8928-8ef6-454d-b6b7-146c41bfec0b None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:39.300130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b8928-8ef6-454d-b6b7-146c41bfec0b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:39.300411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b8928-8ef6-454d-b6b7-146c41bfec0b None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:39.301740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 754/1506] 10.222.0.50 () {52 vars in 830 bytes} [Thu Sep 15 10:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:52:39.305364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:39.306566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:39.409054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f93f7-e0c0-4cbe-ad85-d8667a30ce8e service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:39.502401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:39.503052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:39.503290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:39.503559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] PATH_INFO: `/v3/users/9c45a0f4d90146a98f1f9ede3107ae54` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:39.516237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c45a0f4d90146a98f1f9ede3107ae54)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:39.517281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WARNING py.warnings [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:52:39.517281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: warnings.warn(msg) Sep 15 10:52:39.517281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: Sep 15 10:52:39.518534 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:39.528049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:39.586357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c45a0f4d90146a98f1f9ede3107ae54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:39.586889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c45a0f4d90146a98f1f9ede3107ae54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:39.595933 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c45a0f4d90146a98f1f9ede3107ae54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:39.596366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c45a0f4d90146a98f1f9ede3107ae54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:39.610880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c45a0f4d90146a98f1f9ede3107ae54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:39.638910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 753/1507] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:52:39 2022] DELETE /identity/v3/users/9c45a0f4d90146a98f1f9ede3107ae54 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:52:39.643312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca9998-932e-436f-804f-c10efdacbd9d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:39.643794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca9998-932e-436f-804f-c10efdacbd9d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:39.643907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca9998-932e-436f-804f-c10efdacbd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:39.700678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4dca9998-932e-436f-804f-c10efdacbd9d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:52:39.701850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4dca9998-932e-436f-804f-c10efdacbd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:39.776605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:39.789262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 755/1508] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:52:39.799463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:39.800719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:39.904044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109b6fa-8bdb-4328-8807-c02391bb5c09 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:39.958284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:39.980063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba3242-0940-4720-91f3-cdf3c3b26e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:39.980684 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba3242-0940-4720-91f3-cdf3c3b26e91 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:39.980999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba3242-0940-4720-91f3-cdf3c3b26e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:39.982008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba3242-0940-4720-91f3-cdf3c3b26e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:39.983849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8fba3242-0940-4720-91f3-cdf3c3b26e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:40.094267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba3242-0940-4720-91f3-cdf3c3b26e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:40.095710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba3242-0940-4720-91f3-cdf3c3b26e91 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:40.114173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 754/1509] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:52:40.464105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca9998-932e-436f-804f-c10efdacbd9d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:52:40.465304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4dca9998-932e-436f-804f-c10efdacbd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:52:40.477592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca9998-932e-436f-804f-c10efdacbd9d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:52:40.580241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:52:40.580907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:52:40.581188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:52:40.581459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] PATH_INFO: `/v3/projects/23aec0466e1f42d7a22c7e72c15723f4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:52:40.592400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=23aec0466e1f42d7a22c7e72c15723f4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:52:40.593459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:52:40.593459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 15 10:52:40.593459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: Sep 15 10:52:40.594709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:52:40.653108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23aec0466e1f42d7a22c7e72c15723f4', 'request_id': 'req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:40.659346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23aec0466e1f42d7a22c7e72c15723f4', 'request_id': 'req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:40.668606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23aec0466e1f42d7a22c7e72c15723f4', 'request_id': 'req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:40.698701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] The token cache is being invalidate because project 23aec0466e1f42d7a22c7e72c15723f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:52:40.698984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:40.699499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ccb7c68-e685-40cf-a17d-e7dec0d1aa8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:52:40.701790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 756/1510] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:52:40 2022] DELETE /identity/v3/projects/23aec0466e1f42d7a22c7e72c15723f4 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:52:50.111883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=62 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:52:50.696376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:54:03.448966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbb4c1-f82a-4e43-bbef-82a8e0a5e6a0 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:03.449844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbb4c1-f82a-4e43-bbef-82a8e0a5e6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:03.449844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbb4c1-f82a-4e43-bbef-82a8e0a5e6a0 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:03.451168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:54:03.455939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e976-b746-47c2-b686-286f2519f376 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:03.456372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e976-b746-47c2-b686-286f2519f376 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:03.456488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e976-b746-47c2-b686-286f2519f376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:03.534909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3a75e976-b746-47c2-b686-286f2519f376 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:03.536115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3a75e976-b746-47c2-b686-286f2519f376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:03.642627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:03.797776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbb4c1-f82a-4e43-bbef-82a8e0a5e6a0 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:54:03.799980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-52cbb4c1-f82a-4e43-bbef-82a8e0a5e6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:03.918584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbb4c1-f82a-4e43-bbef-82a8e0a5e6a0 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:54:03.923974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd8424-bd95-4d44-a1f9-dd95f5b4e025 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:54:03.924580 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd8424-bd95-4d44-a1f9-dd95f5b4e025 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:03.924890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd8424-bd95-4d44-a1f9-dd95f5b4e025 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:03.925181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd8424-bd95-4d44-a1f9-dd95f5b4e025 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:03.933442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd8424-bd95-4d44-a1f9-dd95f5b4e025 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:54:03.935062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd8424-bd95-4d44-a1f9-dd95f5b4e025 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:54:03.944557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 756/1513] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:54:05.049200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9d84-a623-4f63-9cac-47224cf7d7f8 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:05.049783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9d84-a623-4f63-9cac-47224cf7d7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:05.049783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9d84-a623-4f63-9cac-47224cf7d7f8 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:05.051165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:54:05.058488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e487d-70ec-428a-b477-0016858931b2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:05.058913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e487d-70ec-428a-b477-0016858931b2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:05.059029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e487d-70ec-428a-b477-0016858931b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:05.079247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-775e487d-70ec-428a-b477-0016858931b2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:05.080435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-775e487d-70ec-428a-b477-0016858931b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:05.088923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:05.243311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb9d84-a623-4f63-9cac-47224cf7d7f8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:54:05.258974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb9d84-a623-4f63-9cac-47224cf7d7f8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:54:05.264570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dcba9-f3f9-4b86-b50e-2ffa59b166aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:54:05.265204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dcba9-f3f9-4b86-b50e-2ffa59b166aa service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:05.265485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dcba9-f3f9-4b86-b50e-2ffa59b166aa service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:05.265754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dcba9-f3f9-4b86-b50e-2ffa59b166aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:05.275165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64dcba9-f3f9-4b86-b50e-2ffa59b166aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:54:05.276931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64dcba9-f3f9-4b86-b50e-2ffa59b166aa service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:54:05.286433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 759/1516] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:54:05.876702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a52e3-2eff-4c04-89ce-e4f37972dc33 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:05.876938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a52e3-2eff-4c04-89ce-e4f37972dc33 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:05.877217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a52e3-2eff-4c04-89ce-e4f37972dc33 None None] PATH_INFO: `/v3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:05.878506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 758/1517] 10.222.0.66 () {54 vars in 925 bytes} [Thu Sep 15 10:54:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:54:05.884984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09c391-26af-469f-af8c-7febf3a42a42 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:05.885274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09c391-26af-469f-af8c-7febf3a42a42 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:05.885551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09c391-26af-469f-af8c-7febf3a42a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:05.945111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2c09c391-26af-469f-af8c-7febf3a42a42 None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:05.946361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2c09c391-26af-469f-af8c-7febf3a42a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:05.963381 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:54:06.020010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 760/1518] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Sep 15 10:54:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:09.570070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f312b81-df0a-4b0e-b59c-107c8d87ba0e None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:09.570547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f312b81-df0a-4b0e-b59c-107c8d87ba0e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:09.570657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f312b81-df0a-4b0e-b59c-107c8d87ba0e None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:09.571943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 759/1519] 10.222.0.50 () {54 vars in 961 bytes} [Thu Sep 15 10:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:54:09.577355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b127b0-9b19-4369-bf7a-1cd0db217c17 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:09.577815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b127b0-9b19-4369-bf7a-1cd0db217c17 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:09.578079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b127b0-9b19-4369-bf7a-1cd0db217c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:09.598325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f9b127b0-9b19-4369-bf7a-1cd0db217c17 None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:09.599509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f9b127b0-9b19-4369-bf7a-1cd0db217c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:09.607889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 761/1520] 10.222.0.50 () {58 vars in 1058 bytes} [Thu Sep 15 10:54:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:09.615428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f312b81-df0a-4b0e-b59c-107c8d87ba0e None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:54:09.629115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f312b81-df0a-4b0e-b59c-107c8d87ba0e None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:54:09.634682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c66ba5-ffd9-421d-a4e3-8a4a97f8fd52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:54:09.635279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66ba5-ffd9-421d-a4e3-8a4a97f8fd52 service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:09.635562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66ba5-ffd9-421d-a4e3-8a4a97f8fd52 service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:09.635835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66ba5-ffd9-421d-a4e3-8a4a97f8fd52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:09.644302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c66ba5-ffd9-421d-a4e3-8a4a97f8fd52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:54:09.645906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c66ba5-ffd9-421d-a4e3-8a4a97f8fd52 service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:54:09.656413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 760/1521] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:54:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:54:16.781040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415680d4-b5a1-4c66-8a54-7751fc5cb6b1 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:16.781649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415680d4-b5a1-4c66-8a54-7751fc5cb6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:16.781649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415680d4-b5a1-4c66-8a54-7751fc5cb6b1 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:16.782954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:54:16.789110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b04da6-c62d-4c23-906d-bd5bc57b5e8f None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:16.789558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b04da6-c62d-4c23-906d-bd5bc57b5e8f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:16.789662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b04da6-c62d-4c23-906d-bd5bc57b5e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:16.809743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-30b04da6-c62d-4c23-906d-bd5bc57b5e8f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:16.810874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-30b04da6-c62d-4c23-906d-bd5bc57b5e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:16.818686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:16.973376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415680d4-b5a1-4c66-8a54-7751fc5cb6b1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:54:16.988162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415680d4-b5a1-4c66-8a54-7751fc5cb6b1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:54:16.993478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49555033-a7d7-4c43-85de-0748e9238d3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:54:16.994067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49555033-a7d7-4c43-85de-0748e9238d3e service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:16.994564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49555033-a7d7-4c43-85de-0748e9238d3e service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:16.995006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49555033-a7d7-4c43-85de-0748e9238d3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:17.002888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49555033-a7d7-4c43-85de-0748e9238d3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:54:17.004464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49555033-a7d7-4c43-85de-0748e9238d3e service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:54:17.014271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 763/1524] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:54:18.084782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424d548-27c8-492f-b832-d0b5f38cefb0 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:18.085385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424d548-27c8-492f-b832-d0b5f38cefb0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:18.085385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424d548-27c8-492f-b832-d0b5f38cefb0 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:18.086709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 10:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:54:18.093585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f02ec-f75f-4b53-be1e-3849569285d5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:18.094032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f02ec-f75f-4b53-be1e-3849569285d5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:18.094137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f02ec-f75f-4b53-be1e-3849569285d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:18.113181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-344f02ec-f75f-4b53-be1e-3849569285d5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:18.114409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-344f02ec-f75f-4b53-be1e-3849569285d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:18.123162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 10:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:18.275128 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424d548-27c8-492f-b832-d0b5f38cefb0 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:54:18.289411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424d548-27c8-492f-b832-d0b5f38cefb0 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:54:18.294691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977486d-5bfa-40d1-8f31-844edeba22af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:54:18.295287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977486d-5bfa-40d1-8f31-844edeba22af service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:18.295564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977486d-5bfa-40d1-8f31-844edeba22af service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:18.295835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977486d-5bfa-40d1-8f31-844edeba22af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:18.303751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977486d-5bfa-40d1-8f31-844edeba22af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:54:18.305374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977486d-5bfa-40d1-8f31-844edeba22af service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:54:18.315401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 763/1527] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:54:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:54:22.301275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ec1ac-78ce-4ba5-bd79-68bad122d757 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:22.301847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ec1ac-78ce-4ba5-bd79-68bad122d757 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:22.301847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ec1ac-78ce-4ba5-bd79-68bad122d757 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:22.303211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:54:22.309202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cf4de-3f7b-424f-98af-282df4adc3e7 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:22.309637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cf4de-3f7b-424f-98af-282df4adc3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:22.309747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cf4de-3f7b-424f-98af-282df4adc3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:22.329384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7a6cf4de-3f7b-424f-98af-282df4adc3e7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:22.330593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7a6cf4de-3f7b-424f-98af-282df4adc3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:22.338504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:22.629836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ec1ac-78ce-4ba5-bd79-68bad122d757 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:54:22.631585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7e4ec1ac-78ce-4ba5-bd79-68bad122d757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:22.708553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ec1ac-78ce-4ba5-bd79-68bad122d757 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:54:22.712049 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd0994-b0ad-4e37-8c64-aa4314a596be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:54:22.712642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd0994-b0ad-4e37-8c64-aa4314a596be service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:22.713162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd0994-b0ad-4e37-8c64-aa4314a596be service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:22.713635 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd0994-b0ad-4e37-8c64-aa4314a596be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:22.723296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd0994-b0ad-4e37-8c64-aa4314a596be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:54:22.724725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd0994-b0ad-4e37-8c64-aa4314a596be service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:54:22.734350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 766/1530] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:54:24.891952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e5e6d-0c1d-417b-ad22-f2e2f1ea50aa None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:24.892553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e5e6d-0c1d-417b-ad22-f2e2f1ea50aa None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:24.892553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e5e6d-0c1d-417b-ad22-f2e2f1ea50aa None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:24.893796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:54:24.898925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f70de9-d76e-4f39-a417-e49a1a8e5ea8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:24.899380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f70de9-d76e-4f39-a417-e49a1a8e5ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:24.899496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f70de9-d76e-4f39-a417-e49a1a8e5ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:24.918989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d5f70de9-d76e-4f39-a417-e49a1a8e5ea8 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:24.920183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d5f70de9-d76e-4f39-a417-e49a1a8e5ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:24.927884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:25.217900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e5e6d-0c1d-417b-ad22-f2e2f1ea50aa None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:54:25.229240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e5e6d-0c1d-417b-ad22-f2e2f1ea50aa None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:54:25.232824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4556f-d902-4b6c-a3b3-b99216e339db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:54:25.233414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4556f-d902-4b6c-a3b3-b99216e339db service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:25.233691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4556f-d902-4b6c-a3b3-b99216e339db service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:25.233958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4556f-d902-4b6c-a3b3-b99216e339db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:25.242677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4556f-d902-4b6c-a3b3-b99216e339db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:54:25.244828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4556f-d902-4b6c-a3b3-b99216e339db service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:54:25.254173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 766/1533] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:54:28.871553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308b6f3-42c1-4498-8056-0668795efe52 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:28.872085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308b6f3-42c1-4498-8056-0668795efe52 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:28.872085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308b6f3-42c1-4498-8056-0668795efe52 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:28.873412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:54:28.878536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd006990-e4df-4bba-a222-99b4b5151a02 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:28.878960 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd006990-e4df-4bba-a222-99b4b5151a02 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:28.879064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd006990-e4df-4bba-a222-99b4b5151a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:28.899747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-bd006990-e4df-4bba-a222-99b4b5151a02 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:28.901084 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-bd006990-e4df-4bba-a222-99b4b5151a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:28.908725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:29.213179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308b6f3-42c1-4498-8056-0668795efe52 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:54:29.214985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0308b6f3-42c1-4498-8056-0668795efe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:29.280158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:54:29.294218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308b6f3-42c1-4498-8056-0668795efe52 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:54:29.297802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572d38e-bf97-41f1-9abd-65572155f7a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:54:29.298410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572d38e-bf97-41f1-9abd-65572155f7a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:29.298690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572d38e-bf97-41f1-9abd-65572155f7a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:29.298962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572d38e-bf97-41f1-9abd-65572155f7a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:29.308343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572d38e-bf97-41f1-9abd-65572155f7a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:54:29.309816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572d38e-bf97-41f1-9abd-65572155f7a5 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:54:29.319052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 769/1536] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:54:31.489037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452975c-8928-4222-92ac-4aaf8264d7fe None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:31.489617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452975c-8928-4222-92ac-4aaf8264d7fe None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:31.489617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452975c-8928-4222-92ac-4aaf8264d7fe None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:31.490930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 10:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:54:31.496242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd539b3-4f4c-46cf-bdef-e6ae8d870f16 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:31.496695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd539b3-4f4c-46cf-bdef-e6ae8d870f16 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:31.496852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd539b3-4f4c-46cf-bdef-e6ae8d870f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:31.516614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7bd539b3-4f4c-46cf-bdef-e6ae8d870f16 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:54:31.517843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7bd539b3-4f4c-46cf-bdef-e6ae8d870f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:31.526120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 10:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:54:31.829394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452975c-8928-4222-92ac-4aaf8264d7fe None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:54:31.831114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4452975c-8928-4222-92ac-4aaf8264d7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:54:31.839338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=84 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:54:31.845404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452975c-8928-4222-92ac-4aaf8264d7fe None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:54:31.849003 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e087e71-980d-458d-8585-723782473d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:54:31.849600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e087e71-980d-458d-8585-723782473d83 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:54:31.849880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e087e71-980d-458d-8585-723782473d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:54:31.850152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e087e71-980d-458d-8585-723782473d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:54:31.859669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e087e71-980d-458d-8585-723782473d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:54:31.861136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e087e71-980d-458d-8585-723782473d83 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:54:31.870526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 769/1539] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:54:41.511815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=11 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:54:41.866314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:55.599407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75aed-db79-42b8-9c91-860d7fe2879f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:55.600092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75aed-db79-42b8-9c91-860d7fe2879f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:55.600092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75aed-db79-42b8-9c91-860d7fe2879f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:55.629552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-08a75aed-db79-42b8-9c91-860d7fe2879f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:55.630834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-08a75aed-db79-42b8-9c91-860d7fe2879f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:55.643221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 771/1540] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:55.653107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1f818-1bfe-4139-a052-c4156ef31260 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:55.654174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1f818-1bfe-4139-a052-c4156ef31260 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:55.654686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1f818-1bfe-4139-a052-c4156ef31260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:55.684589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-87a1f818-1bfe-4139-a052-c4156ef31260 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:55.686285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-87a1f818-1bfe-4139-a052-c4156ef31260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:55.701285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 770/1541] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:55.706043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a75aed-db79-42b8-9c91-860d7fe2879f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:55.726866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a75aed-db79-42b8-9c91-860d7fe2879f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:55.732615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da8509-aed5-47e3-b135-28d0b524d7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:55.733281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da8509-aed5-47e3-b135-28d0b524d7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:55.733566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da8509-aed5-47e3-b135-28d0b524d7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:55.733846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da8509-aed5-47e3-b135-28d0b524d7e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:55.734506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da8509-aed5-47e3-b135-28d0b524d7e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:55:55.734808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da8509-aed5-47e3-b135-28d0b524d7e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:55.736451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da8509-aed5-47e3-b135-28d0b524d7e4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:55.762790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 772/1542] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:55:55.772708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1f818-1bfe-4139-a052-c4156ef31260 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:55.786356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1f818-1bfe-4139-a052-c4156ef31260 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:55.795532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172f9b8-8d2b-42f4-bccf-29197cc9601b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:55.796488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172f9b8-8d2b-42f4-bccf-29197cc9601b admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:55.797015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172f9b8-8d2b-42f4-bccf-29197cc9601b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:55.798107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172f9b8-8d2b-42f4-bccf-29197cc9601b admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:55.799059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172f9b8-8d2b-42f4-bccf-29197cc9601b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:55.801676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172f9b8-8d2b-42f4-bccf-29197cc9601b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:55.855892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 771/1543] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:55 2022] POST /identity/v3/projects => generated 365 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:55.860399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da8509-aed5-47e3-b135-28d0b524d7e4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:55.875787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da8509-aed5-47e3-b135-28d0b524d7e4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:55.918171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a5f5e-bc7e-4460-a860-3a83e798a965 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:55.918763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a5f5e-bc7e-4460-a860-3a83e798a965 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:55.919354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a5f5e-bc7e-4460-a860-3a83e798a965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:55.951380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-054a5f5e-bc7e-4460-a860-3a83e798a965 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:55.952946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-054a5f5e-bc7e-4460-a860-3a83e798a965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.001124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bf0b4-5f0a-4270-8e7e-8ade16aa82d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.001777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bf0b4-5f0a-4270-8e7e-8ade16aa82d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.002043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bf0b4-5f0a-4270-8e7e-8ade16aa82d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.002302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bf0b4-5f0a-4270-8e7e-8ade16aa82d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.002951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bf0b4-5f0a-4270-8e7e-8ade16aa82d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.007685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bf0b4-5f0a-4270-8e7e-8ade16aa82d1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.046741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 773/1544] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:55 2022] POST /identity/v3/users => generated 283 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:55:56.048099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69aead-0341-4073-bc82-8ef3baf6e7d0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.048388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69aead-0341-4073-bc82-8ef3baf6e7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.048666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69aead-0341-4073-bc82-8ef3baf6e7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.063363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 772/1545] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:55:56.064948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa55ee-16ad-4468-9e0d-7c88280b8ce9 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.065796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa55ee-16ad-4468-9e0d-7c88280b8ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.066245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa55ee-16ad-4468-9e0d-7c88280b8ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.081370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1d69aead-0341-4073-bc82-8ef3baf6e7d0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.082599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1d69aead-0341-4073-bc82-8ef3baf6e7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.096618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-49fa55ee-16ad-4468-9e0d-7c88280b8ce9 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.098133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-49fa55ee-16ad-4468-9e0d-7c88280b8ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.104237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 774/1546] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.106765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69aead-0341-4073-bc82-8ef3baf6e7d0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.122161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 773/1547] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.123840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9196f-acd0-4a2d-8a38-ebdf35aeb9be None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.124284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9196f-acd0-4a2d-8a38-ebdf35aeb9be None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.124732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9196f-acd0-4a2d-8a38-ebdf35aeb9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.134548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69aead-0341-4073-bc82-8ef3baf6e7d0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.146225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f547a-6db5-4098-bc7b-b33f0992abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.152938 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f547a-6db5-4098-bc7b-b33f0992abc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.153571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f547a-6db5-4098-bc7b-b33f0992abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.154029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f547a-6db5-4098-bc7b-b33f0992abc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.154859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f547a-6db5-4098-bc7b-b33f0992abc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.156685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f547a-6db5-4098-bc7b-b33f0992abc3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.170318 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-f0c9196f-acd0-4a2d-8a38-ebdf35aeb9be None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.171825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f0c9196f-acd0-4a2d-8a38-ebdf35aeb9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.176889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 775/1548] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:56.176889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2d9f8-32ff-4293-8235-74187bef36ad None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.176889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2d9f8-32ff-4293-8235-74187bef36ad None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.176889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2d9f8-32ff-4293-8235-74187bef36ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.190163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 774/1549] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.191525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aaf8c-4572-4c3e-b3ad-b3c1b07d67a6 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.191812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aaf8c-4572-4c3e-b3ad-b3c1b07d67a6 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.192087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aaf8c-4572-4c3e-b3ad-b3c1b07d67a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.200031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6ff2d9f8-32ff-4293-8235-74187bef36ad None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.201304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6ff2d9f8-32ff-4293-8235-74187bef36ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.209779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 776/1550] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.210840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681a07-b0c4-41b3-ab45-d8804a022a14 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.211138 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681a07-b0c4-41b3-ab45-d8804a022a14 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.211414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681a07-b0c4-41b3-ab45-d8804a022a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.214441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9d5aaf8c-4572-4c3e-b3ad-b3c1b07d67a6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.215785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9d5aaf8c-4572-4c3e-b3ad-b3c1b07d67a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.225607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 775/1551] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.226868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912994e-08a0-473a-a6c0-865e15fa99f5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.227196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912994e-08a0-473a-a6c0-865e15fa99f5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.228009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912994e-08a0-473a-a6c0-865e15fa99f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.234187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cc681a07-b0c4-41b3-ab45-d8804a022a14 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.235426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cc681a07-b0c4-41b3-ab45-d8804a022a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.243745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 777/1552] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.244493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681a07-b0c4-41b3-ab45-d8804a022a14 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.252404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4912994e-08a0-473a-a6c0-865e15fa99f5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.253743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4912994e-08a0-473a-a6c0-865e15fa99f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.254258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681a07-b0c4-41b3-ab45-d8804a022a14 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.260671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdcad3-172f-45cc-9cf2-d61aa6c9fa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.261311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdcad3-172f-45cc-9cf2-d61aa6c9fa43 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.261588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdcad3-172f-45cc-9cf2-d61aa6c9fa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.261859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdcad3-172f-45cc-9cf2-d61aa6c9fa43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.262494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdcad3-172f-45cc-9cf2-d61aa6c9fa43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.264116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdcad3-172f-45cc-9cf2-d61aa6c9fa43 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.269924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 776/1553] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.271330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec6982-c8dd-4275-b34b-cde48ab33670 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.271624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec6982-c8dd-4275-b34b-cde48ab33670 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.271910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec6982-c8dd-4275-b34b-cde48ab33670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.276778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 778/1554] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:56.277682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdcad3-172f-45cc-9cf2-d61aa6c9fa43 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.289318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdcad3-172f-45cc-9cf2-d61aa6c9fa43 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.294770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953527c-7c0e-491d-95aa-e8f8301f109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.295364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953527c-7c0e-491d-95aa-e8f8301f109c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.295642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953527c-7c0e-491d-95aa-e8f8301f109c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.295912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953527c-7c0e-491d-95aa-e8f8301f109c admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.296538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953527c-7c0e-491d-95aa-e8f8301f109c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:55:56.296857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953527c-7c0e-491d-95aa-e8f8301f109c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.296976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-c7ec6982-c8dd-4275-b34b-cde48ab33670 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.298177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c7ec6982-c8dd-4275-b34b-cde48ab33670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.298458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953527c-7c0e-491d-95aa-e8f8301f109c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.306767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 777/1555] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.308562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018479ed-a749-4c39-be49-d0f044a13e72 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.308879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018479ed-a749-4c39-be49-d0f044a13e72 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.309336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018479ed-a749-4c39-be49-d0f044a13e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.311352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 779/1556] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:55:56.312087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953527c-7c0e-491d-95aa-e8f8301f109c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.322390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953527c-7c0e-491d-95aa-e8f8301f109c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.327391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad3c6c-8169-4d58-95c7-1f2f5b03603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.328029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad3c6c-8169-4d58-95c7-1f2f5b03603c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.328315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad3c6c-8169-4d58-95c7-1f2f5b03603c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.328581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad3c6c-8169-4d58-95c7-1f2f5b03603c admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.329235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad3c6c-8169-4d58-95c7-1f2f5b03603c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:55:56.329536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad3c6c-8169-4d58-95c7-1f2f5b03603c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.331130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad3c6c-8169-4d58-95c7-1f2f5b03603c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.335112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-018479ed-a749-4c39-be49-d0f044a13e72 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.336348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-018479ed-a749-4c39-be49-d0f044a13e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.343266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 780/1557] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:55:56.344042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad3c6c-8169-4d58-95c7-1f2f5b03603c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.354029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 778/1558] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.355309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d42bf-6bd6-42da-8050-8c9f2fb30dd1 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.355599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d42bf-6bd6-42da-8050-8c9f2fb30dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.355872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d42bf-6bd6-42da-8050-8c9f2fb30dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.358843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad3c6c-8169-4d58-95c7-1f2f5b03603c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.363430 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138a862-8564-41c1-b7d2-9b0527bc711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.364056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138a862-8564-41c1-b7d2-9b0527bc711e admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.364338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138a862-8564-41c1-b7d2-9b0527bc711e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.364632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138a862-8564-41c1-b7d2-9b0527bc711e admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.365296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138a862-8564-41c1-b7d2-9b0527bc711e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:55:56.365594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138a862-8564-41c1-b7d2-9b0527bc711e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.367236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138a862-8564-41c1-b7d2-9b0527bc711e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.375439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-0f8d42bf-6bd6-42da-8050-8c9f2fb30dd1 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.376657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0f8d42bf-6bd6-42da-8050-8c9f2fb30dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.379211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 781/1559] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:55:56.380482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bea61-6f7b-4cff-a23d-4f3fd801ca66 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.380757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bea61-6f7b-4cff-a23d-4f3fd801ca66 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.381083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bea61-6f7b-4cff-a23d-4f3fd801ca66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.392278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 779/1560] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.393154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d42bf-6bd6-42da-8050-8c9f2fb30dd1 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.404945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-141bea61-6f7b-4cff-a23d-4f3fd801ca66 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.405118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d42bf-6bd6-42da-8050-8c9f2fb30dd1 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.406154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-141bea61-6f7b-4cff-a23d-4f3fd801ca66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.409564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a93a33-93a9-49e7-be46-24d63aac1bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.410221 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a93a33-93a9-49e7-be46-24d63aac1bab admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.410504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a93a33-93a9-49e7-be46-24d63aac1bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.410778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a93a33-93a9-49e7-be46-24d63aac1bab admin admin] PATH_INFO: `/v3/projects/63a6561e6952440da97f5f22dec2799c/users/20c22261ce094adbad87694311fe0368/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.415312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 782/1561] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.416601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a0457-e609-4acc-a00e-9aff6f6fb51c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.417007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a0457-e609-4acc-a00e-9aff6f6fb51c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.417288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a0457-e609-4acc-a00e-9aff6f6fb51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.427138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a93a33-93a9-49e7-be46-24d63aac1bab admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a6561e6952440da97f5f22dec2799c, user_id=20c22261ce094adbad87694311fe0368, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.429886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a93a33-93a9-49e7-be46-24d63aac1bab admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.436919 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-624a0457-e609-4acc-a00e-9aff6f6fb51c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.438219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-624a0457-e609-4acc-a00e-9aff6f6fb51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.443732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 780/1562] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:56 2022] PUT /identity/v3/projects/63a6561e6952440da97f5f22dec2799c/users/20c22261ce094adbad87694311fe0368/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:56.455931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a93a33-93a9-49e7-be46-24d63aac1bab admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.479864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:56.501882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 783/1563] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.502893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a0457-e609-4acc-a00e-9aff6f6fb51c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.504379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:56.513190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a0457-e609-4acc-a00e-9aff6f6fb51c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.515978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a93a33-93a9-49e7-be46-24d63aac1bab admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.518346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c5a59-2515-4e3b-b1fe-0d45790cc74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.518952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c5a59-2515-4e3b-b1fe-0d45790cc74e admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.519238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c5a59-2515-4e3b-b1fe-0d45790cc74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.519511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c5a59-2515-4e3b-b1fe-0d45790cc74e admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.520138 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c5a59-2515-4e3b-b1fe-0d45790cc74e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.521200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-025c5a59-2515-4e3b-b1fe-0d45790cc74e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:55:56.521200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 15 10:55:56.521200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: Sep 15 10:55:56.521504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdbdb2-4ed4-4507-91e1-d26570905454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.521848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdbdb2-4ed4-4507-91e1-d26570905454 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.522126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdbdb2-4ed4-4507-91e1-d26570905454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.522394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdbdb2-4ed4-4507-91e1-d26570905454 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.522693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c5a59-2515-4e3b-b1fe-0d45790cc74e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.523022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccdbdb2-4ed4-4507-91e1-d26570905454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.525615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccdbdb2-4ed4-4507-91e1-d26570905454 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.553897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 781/1564] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:56.553897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdbdb2-4ed4-4507-91e1-d26570905454 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.558749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 784/1565] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:56.559614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c5a59-2515-4e3b-b1fe-0d45790cc74e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.621911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdbdb2-4ed4-4507-91e1-d26570905454 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.622114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c5a59-2515-4e3b-b1fe-0d45790cc74e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.626799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd67b89-4d61-45bb-823f-316b7160fa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.626983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c865bb-ae60-404a-ae02-f18b4d949020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.627445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c865bb-ae60-404a-ae02-f18b4d949020 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.627548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd67b89-4d61-45bb-823f-316b7160fa73 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.627759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c865bb-ae60-404a-ae02-f18b4d949020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.627898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd67b89-4d61-45bb-823f-316b7160fa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.628004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c865bb-ae60-404a-ae02-f18b4d949020 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.628106 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd67b89-4d61-45bb-823f-316b7160fa73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.628639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c865bb-ae60-404a-ae02-f18b4d949020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:55:56.628773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd67b89-4d61-45bb-823f-316b7160fa73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.628979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c865bb-ae60-404a-ae02-f18b4d949020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.629994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WARNING py.warnings [None req-53c865bb-ae60-404a-ae02-f18b4d949020 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:55:56.629994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: warnings.warn(msg) Sep 15 10:55:56.629994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: Sep 15 10:55:56.630677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd67b89-4d61-45bb-823f-316b7160fa73 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.630927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c865bb-ae60-404a-ae02-f18b4d949020 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.643300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 782/1566] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:55:56.644504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4b81c-e41b-4f24-b265-ea47591194b0 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.644792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4b81c-e41b-4f24-b265-ea47591194b0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.645097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4b81c-e41b-4f24-b265-ea47591194b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.665295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-45c4b81c-e41b-4f24-b265-ea47591194b0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.666554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-45c4b81c-e41b-4f24-b265-ea47591194b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.666669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 785/1567] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:56.669813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd67b89-4d61-45bb-823f-316b7160fa73 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.731484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 783/1568] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.732833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76ab73-fcb8-48e9-a48b-b81f6085eb6d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.733121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76ab73-fcb8-48e9-a48b-b81f6085eb6d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.733393 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76ab73-fcb8-48e9-a48b-b81f6085eb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.734527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd67b89-4d61-45bb-823f-316b7160fa73 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.739208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fa564-208b-4574-9fb9-f71b9f33f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.739820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fa564-208b-4574-9fb9-f71b9f33f747 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.740234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fa564-208b-4574-9fb9-f71b9f33f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.740514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fa564-208b-4574-9fb9-f71b9f33f747 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.741180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fa564-208b-4574-9fb9-f71b9f33f747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:55:56.741486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fa564-208b-4574-9fb9-f71b9f33f747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.743877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fa564-208b-4574-9fb9-f71b9f33f747 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.756002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 786/1569] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:55:56.756850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fa564-208b-4574-9fb9-f71b9f33f747 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.767360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fa564-208b-4574-9fb9-f71b9f33f747 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.767575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-bf76ab73-fcb8-48e9-a48b-b81f6085eb6d None None] MFA Rules not processed for user `20c22261ce094adbad87694311fe0368`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:56.768893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-bf76ab73-fcb8-48e9-a48b-b81f6085eb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:56.771998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ece91-078f-43ec-91af-db343c08f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.772607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ece91-078f-43ec-91af-db343c08f4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.772921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ece91-078f-43ec-91af-db343c08f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.773218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ece91-078f-43ec-91af-db343c08f4f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.773907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ece91-078f-43ec-91af-db343c08f4f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:55:56.774431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ece91-078f-43ec-91af-db343c08f4f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.776107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ece91-078f-43ec-91af-db343c08f4f5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.788098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 787/1570] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:55:56.789143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ece91-078f-43ec-91af-db343c08f4f5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.799995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ece91-078f-43ec-91af-db343c08f4f5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.804606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccaa2ef-b4af-4d5e-b1d7-d4364ab0df8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.805799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccaa2ef-b4af-4d5e-b1d7-d4364ab0df8f admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.806115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccaa2ef-b4af-4d5e-b1d7-d4364ab0df8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.806363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccaa2ef-b4af-4d5e-b1d7-d4364ab0df8f admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.806991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccaa2ef-b4af-4d5e-b1d7-d4364ab0df8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.808966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccaa2ef-b4af-4d5e-b1d7-d4364ab0df8f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.833020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 788/1571] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:56.833859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccaa2ef-b4af-4d5e-b1d7-d4364ab0df8f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.837415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 784/1572] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:56.838299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76ab73-fcb8-48e9-a48b-b81f6085eb6d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.843570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:56.846447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccaa2ef-b4af-4d5e-b1d7-d4364ab0df8f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.849910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76ab73-fcb8-48e9-a48b-b81f6085eb6d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.851295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d2546-d211-45eb-947a-beb4bc9bf270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.851915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d2546-d211-45eb-947a-beb4bc9bf270 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.852339 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d2546-d211-45eb-947a-beb4bc9bf270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.852663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d2546-d211-45eb-947a-beb4bc9bf270 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.853331 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d2546-d211-45eb-947a-beb4bc9bf270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.855220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e70eeb-f984-42cf-84f6-1c69a734b0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.855368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d2546-d211-45eb-947a-beb4bc9bf270 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.855818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e70eeb-f984-42cf-84f6-1c69a734b0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.856098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e70eeb-f984-42cf-84f6-1c69a734b0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.856371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e70eeb-f984-42cf-84f6-1c69a734b0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.857018 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e70eeb-f984-42cf-84f6-1c69a734b0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.858927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e70eeb-f984-42cf-84f6-1c69a734b0b4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.865214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:56.879950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 789/1573] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:56.880691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d2546-d211-45eb-947a-beb4bc9bf270 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.892481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 785/1574] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:56.892481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e70eeb-f984-42cf-84f6-1c69a734b0b4 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.893340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d2546-d211-45eb-947a-beb4bc9bf270 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.953259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e70eeb-f984-42cf-84f6-1c69a734b0b4 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.954181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dc6be-8a3f-46ad-973a-b1e4a5f93dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.954803 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dc6be-8a3f-46ad-973a-b1e4a5f93dde admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.955087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dc6be-8a3f-46ad-973a-b1e4a5f93dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.955354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dc6be-8a3f-46ad-973a-b1e4a5f93dde admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.955978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dc6be-8a3f-46ad-973a-b1e4a5f93dde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.957964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86f3e2-47f6-4a3e-b4dd-edb0dd3d27fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.958102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dc6be-8a3f-46ad-973a-b1e4a5f93dde admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.958572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86f3e2-47f6-4a3e-b4dd-edb0dd3d27fb admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.958852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86f3e2-47f6-4a3e-b4dd-edb0dd3d27fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.959123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86f3e2-47f6-4a3e-b4dd-edb0dd3d27fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.959761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86f3e2-47f6-4a3e-b4dd-edb0dd3d27fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:55:56.960063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86f3e2-47f6-4a3e-b4dd-edb0dd3d27fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.961806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86f3e2-47f6-4a3e-b4dd-edb0dd3d27fb admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:56.974878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 786/1575] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:55:56.976131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86f3e2-47f6-4a3e-b4dd-edb0dd3d27fb admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.980204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 790/1576] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:56.980926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dc6be-8a3f-46ad-973a-b1e4a5f93dde admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:56.991157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86f3e2-47f6-4a3e-b4dd-edb0dd3d27fb admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.994304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dc6be-8a3f-46ad-973a-b1e4a5f93dde admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:56.996735 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395120a8-ac87-4f70-96c1-cdfc4a377aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.997445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395120a8-ac87-4f70-96c1-cdfc4a377aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.997734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395120a8-ac87-4f70-96c1-cdfc4a377aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:56.998058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395120a8-ac87-4f70-96c1-cdfc4a377aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:56.998705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395120a8-ac87-4f70-96c1-cdfc4a377aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:56.998872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80393afb-8653-40e9-8793-8e066f5eeba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:56.999470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80393afb-8653-40e9-8793-8e066f5eeba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:56.999749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80393afb-8653-40e9-8793-8e066f5eeba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.000018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80393afb-8653-40e9-8793-8e066f5eeba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.000636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80393afb-8653-40e9-8793-8e066f5eeba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.000740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395120a8-ac87-4f70-96c1-cdfc4a377aa6 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.002564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80393afb-8653-40e9-8793-8e066f5eeba5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.033010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 787/1577] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:57.033010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395120a8-ac87-4f70-96c1-cdfc4a377aa6 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.033318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 791/1578] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:56 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:57.033318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80393afb-8653-40e9-8793-8e066f5eeba5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.094959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395120a8-ac87-4f70-96c1-cdfc4a377aa6 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.096598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80393afb-8653-40e9-8793-8e066f5eeba5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.099662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d13b5-21eb-4b6d-a76e-8d05f1638628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.100267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d13b5-21eb-4b6d-a76e-8d05f1638628 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.100550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d13b5-21eb-4b6d-a76e-8d05f1638628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.100857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d13b5-21eb-4b6d-a76e-8d05f1638628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.101168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a9aaa-f2e5-4bed-9cfc-25594768b046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.101503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d13b5-21eb-4b6d-a76e-8d05f1638628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.101792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a9aaa-f2e5-4bed-9cfc-25594768b046 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.102069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a9aaa-f2e5-4bed-9cfc-25594768b046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.102875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a9aaa-f2e5-4bed-9cfc-25594768b046 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.103673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a9aaa-f2e5-4bed-9cfc-25594768b046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.103795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d13b5-21eb-4b6d-a76e-8d05f1638628 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.105606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a9aaa-f2e5-4bed-9cfc-25594768b046 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.114064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 788/1579] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.114894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d13b5-21eb-4b6d-a76e-8d05f1638628 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.124553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d13b5-21eb-4b6d-a76e-8d05f1638628 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.129364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26db0d0-cc77-49e7-a247-c72906a87c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.130028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26db0d0-cc77-49e7-a247-c72906a87c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.130248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26db0d0-cc77-49e7-a247-c72906a87c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.130537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26db0d0-cc77-49e7-a247-c72906a87c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.131160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26db0d0-cc77-49e7-a247-c72906a87c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.132780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26db0d0-cc77-49e7-a247-c72906a87c21 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.138038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 792/1580] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:57.138038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a9aaa-f2e5-4bed-9cfc-25594768b046 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.143592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 789/1581] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.144394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26db0d0-cc77-49e7-a247-c72906a87c21 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.154759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26db0d0-cc77-49e7-a247-c72906a87c21 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.185030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:57.191450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:57.203309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a9aaa-f2e5-4bed-9cfc-25594768b046 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.208197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5cae8-77d0-4c2b-b386-a6c90e9a6168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.208823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5cae8-77d0-4c2b-b386-a6c90e9a6168 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.209121 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5cae8-77d0-4c2b-b386-a6c90e9a6168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.209390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5cae8-77d0-4c2b-b386-a6c90e9a6168 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.210021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5cae8-77d0-4c2b-b386-a6c90e9a6168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.212044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5cae8-77d0-4c2b-b386-a6c90e9a6168 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.219075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c2e93-98ee-4608-9c34-ebe5cccf2ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.219757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c2e93-98ee-4608-9c34-ebe5cccf2ccd admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.220166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c2e93-98ee-4608-9c34-ebe5cccf2ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.220444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c2e93-98ee-4608-9c34-ebe5cccf2ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.221666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c2e93-98ee-4608-9c34-ebe5cccf2ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.223839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c2e93-98ee-4608-9c34-ebe5cccf2ccd admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.233716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 793/1582] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:57.237867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5cae8-77d0-4c2b-b386-a6c90e9a6168 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.248281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5cae8-77d0-4c2b-b386-a6c90e9a6168 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.252985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 790/1583] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:57.253145 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acad4c2-7c6d-40a8-8b7e-9a5edd116dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.253730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acad4c2-7c6d-40a8-8b7e-9a5edd116dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.253841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c2e93-98ee-4608-9c34-ebe5cccf2ccd admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.254108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acad4c2-7c6d-40a8-8b7e-9a5edd116dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.254321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acad4c2-7c6d-40a8-8b7e-9a5edd116dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.254926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acad4c2-7c6d-40a8-8b7e-9a5edd116dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.256544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acad4c2-7c6d-40a8-8b7e-9a5edd116dbf admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.267995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 794/1584] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.268662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acad4c2-7c6d-40a8-8b7e-9a5edd116dbf admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.278456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acad4c2-7c6d-40a8-8b7e-9a5edd116dbf admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.323094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c2e93-98ee-4608-9c34-ebe5cccf2ccd admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.328289 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9059a1-7467-4397-b51d-e50df17dbdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.328991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9059a1-7467-4397-b51d-e50df17dbdeb admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.329281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9059a1-7467-4397-b51d-e50df17dbdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.329819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9059a1-7467-4397-b51d-e50df17dbdeb admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.330242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9059a1-7467-4397-b51d-e50df17dbdeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.331262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WARNING py.warnings [None req-9b9059a1-7467-4397-b51d-e50df17dbdeb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:55:57.331262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: warnings.warn(msg) Sep 15 10:55:57.331262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: Sep 15 10:55:57.332531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9059a1-7467-4397-b51d-e50df17dbdeb admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.335847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9f050-1e32-4234-8ec2-d84b8f49e135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.336463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9f050-1e32-4234-8ec2-d84b8f49e135 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.336858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9f050-1e32-4234-8ec2-d84b8f49e135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.337134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9f050-1e32-4234-8ec2-d84b8f49e135 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.337912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9f050-1e32-4234-8ec2-d84b8f49e135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.339837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9f050-1e32-4234-8ec2-d84b8f49e135 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.355589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 791/1585] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:57.355841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9059a1-7467-4397-b51d-e50df17dbdeb admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.365559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9059a1-7467-4397-b51d-e50df17dbdeb admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.365749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 795/1586] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:57.366392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9f050-1e32-4234-8ec2-d84b8f49e135 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.371074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86fb45-dec4-43c4-b94f-da4d3e457d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.371682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86fb45-dec4-43c4-b94f-da4d3e457d9e admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.372046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86fb45-dec4-43c4-b94f-da4d3e457d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.372244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86fb45-dec4-43c4-b94f-da4d3e457d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.372891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86fb45-dec4-43c4-b94f-da4d3e457d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.374497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86fb45-dec4-43c4-b94f-da4d3e457d9e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.377909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9f050-1e32-4234-8ec2-d84b8f49e135 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.383109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc208d76-a445-433d-a55a-68c956434baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.383701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc208d76-a445-433d-a55a-68c956434baa admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.384196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc208d76-a445-433d-a55a-68c956434baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.384484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc208d76-a445-433d-a55a-68c956434baa admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.385147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc208d76-a445-433d-a55a-68c956434baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.388540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 792/1587] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.388540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86fb45-dec4-43c4-b94f-da4d3e457d9e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.388752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc208d76-a445-433d-a55a-68c956434baa admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.395811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86fb45-dec4-43c4-b94f-da4d3e457d9e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.400825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa05e8-3038-4639-bb42-f4c232c4310f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.401417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa05e8-3038-4639-bb42-f4c232c4310f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.401699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa05e8-3038-4639-bb42-f4c232c4310f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.401972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa05e8-3038-4639-bb42-f4c232c4310f admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.402602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa05e8-3038-4639-bb42-f4c232c4310f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.404239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa05e8-3038-4639-bb42-f4c232c4310f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.408676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 796/1588] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:57.412411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc208d76-a445-433d-a55a-68c956434baa admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.418030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 793/1589] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.418030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa05e8-3038-4639-bb42-f4c232c4310f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.423674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc208d76-a445-433d-a55a-68c956434baa admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.425916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa05e8-3038-4639-bb42-f4c232c4310f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.428461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704447e1-ebc0-433c-9ebd-838e0d3437d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.429132 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704447e1-ebc0-433c-9ebd-838e0d3437d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.429424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704447e1-ebc0-433c-9ebd-838e0d3437d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.429686 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704447e1-ebc0-433c-9ebd-838e0d3437d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.430400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704447e1-ebc0-433c-9ebd-838e0d3437d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.431283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d2f9-d836-473e-aadb-99a5aff0925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.431877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d2f9-d836-473e-aadb-99a5aff0925c admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.432158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d2f9-d836-473e-aadb-99a5aff0925c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.432432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d2f9-d836-473e-aadb-99a5aff0925c admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.433048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704447e1-ebc0-433c-9ebd-838e0d3437d1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.433161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3d2f9-d836-473e-aadb-99a5aff0925c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.435008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3d2f9-d836-473e-aadb-99a5aff0925c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.444316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 797/1590] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.445155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704447e1-ebc0-433c-9ebd-838e0d3437d1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.455858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704447e1-ebc0-433c-9ebd-838e0d3437d1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.458178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 794/1591] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:57.458857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d2f9-d836-473e-aadb-99a5aff0925c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.464580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7036744-01ff-4e5e-94ed-5caaa479a3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.465215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7036744-01ff-4e5e-94ed-5caaa479a3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.465495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7036744-01ff-4e5e-94ed-5caaa479a3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.465764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7036744-01ff-4e5e-94ed-5caaa479a3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.466402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7036744-01ff-4e5e-94ed-5caaa479a3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.468018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7036744-01ff-4e5e-94ed-5caaa479a3b0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.469099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d2f9-d836-473e-aadb-99a5aff0925c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.473673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23a3ef-d609-4a2e-acd8-9e3f71fb7544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.474314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23a3ef-d609-4a2e-acd8-9e3f71fb7544 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.474593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23a3ef-d609-4a2e-acd8-9e3f71fb7544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.474859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23a3ef-d609-4a2e-acd8-9e3f71fb7544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.475480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23a3ef-d609-4a2e-acd8-9e3f71fb7544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.477112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23a3ef-d609-4a2e-acd8-9e3f71fb7544 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.479439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 798/1592] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.480313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7036744-01ff-4e5e-94ed-5caaa479a3b0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.488238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 795/1593] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.489046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23a3ef-d609-4a2e-acd8-9e3f71fb7544 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.492751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7036744-01ff-4e5e-94ed-5caaa479a3b0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.498141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83070804-b9b1-403e-8309-825e72d48cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.498386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23a3ef-d609-4a2e-acd8-9e3f71fb7544 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.498745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83070804-b9b1-403e-8309-825e72d48cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.499076 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83070804-b9b1-403e-8309-825e72d48cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.499337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83070804-b9b1-403e-8309-825e72d48cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.499974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83070804-b9b1-403e-8309-825e72d48cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.501643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83070804-b9b1-403e-8309-825e72d48cf9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.503013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cf3c2-8b92-4511-9156-13d5f19d38a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.503654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cf3c2-8b92-4511-9156-13d5f19d38a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.503934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cf3c2-8b92-4511-9156-13d5f19d38a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.504208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cf3c2-8b92-4511-9156-13d5f19d38a0 admin admin] PATH_INFO: `/v3/projects/65c8194f4a0a45e1888412dc0999e547/users/1eeafdaa1db4436abac229a6f48543a9/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.511969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 799/1594] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.512757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83070804-b9b1-403e-8309-825e72d48cf9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.519404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cf3c2-8b92-4511-9156-13d5f19d38a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c8194f4a0a45e1888412dc0999e547, user_id=1eeafdaa1db4436abac229a6f48543a9, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.522111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cf3c2-8b92-4511-9156-13d5f19d38a0 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.522286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83070804-b9b1-403e-8309-825e72d48cf9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.528583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b98a5-9f0e-4629-aac4-1dfd2f212448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.529297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b98a5-9f0e-4629-aac4-1dfd2f212448 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.529578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b98a5-9f0e-4629-aac4-1dfd2f212448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.529851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b98a5-9f0e-4629-aac4-1dfd2f212448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.530594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b98a5-9f0e-4629-aac4-1dfd2f212448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.532588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b98a5-9f0e-4629-aac4-1dfd2f212448 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.538325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 796/1595] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:57 2022] PUT /identity/v3/projects/65c8194f4a0a45e1888412dc0999e547/users/1eeafdaa1db4436abac229a6f48543a9/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:57.539102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cf3c2-8b92-4511-9156-13d5f19d38a0 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.551010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 800/1596] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.551010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b98a5-9f0e-4629-aac4-1dfd2f212448 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.559066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b98a5-9f0e-4629-aac4-1dfd2f212448 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.584971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:57.607105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:57.608335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cf3c2-8b92-4511-9156-13d5f19d38a0 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.613462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29b4cf-1879-4972-b683-f6617ac6e4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.614127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29b4cf-1879-4972-b683-f6617ac6e4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.614406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29b4cf-1879-4972-b683-f6617ac6e4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.614683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29b4cf-1879-4972-b683-f6617ac6e4b5 admin admin] PATH_INFO: `/v3/projects/5a4cf92cfae0440297266ef8e1d501fc/users/e1fdbf0fb6b14090bf38fac073d01814/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.619707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47175181-49ea-45e7-891d-c35691130d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.620328 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175181-49ea-45e7-891d-c35691130d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.620613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175181-49ea-45e7-891d-c35691130d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.620924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175181-49ea-45e7-891d-c35691130d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.621564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47175181-49ea-45e7-891d-c35691130d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.623227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47175181-49ea-45e7-891d-c35691130d15 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.629852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29b4cf-1879-4972-b683-f6617ac6e4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4cf92cfae0440297266ef8e1d501fc, user_id=e1fdbf0fb6b14090bf38fac073d01814, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.632647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29b4cf-1879-4972-b683-f6617ac6e4b5 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.633435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 801/1597] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.634306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47175181-49ea-45e7-891d-c35691130d15 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.646234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 797/1598] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:57 2022] PUT /identity/v3/projects/5a4cf92cfae0440297266ef8e1d501fc/users/e1fdbf0fb6b14090bf38fac073d01814/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:57.646416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47175181-49ea-45e7-891d-c35691130d15 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.646968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29b4cf-1879-4972-b683-f6617ac6e4b5 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.708479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f83b68-68a2-4f15-80c4-7da882809823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.709114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f83b68-68a2-4f15-80c4-7da882809823 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.709252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29b4cf-1879-4972-b683-f6617ac6e4b5 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.709708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f83b68-68a2-4f15-80c4-7da882809823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.709708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f83b68-68a2-4f15-80c4-7da882809823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.710283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f83b68-68a2-4f15-80c4-7da882809823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.712564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f83b68-68a2-4f15-80c4-7da882809823 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.714671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcdfe6-d282-452d-ac57-94f35ae7947c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.715337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcdfe6-d282-452d-ac57-94f35ae7947c admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.715618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcdfe6-d282-452d-ac57-94f35ae7947c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.715891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcdfe6-d282-452d-ac57-94f35ae7947c admin admin] PATH_INFO: `/v3/projects/8645ab107c9b4d8bb4015aa8529c6ea5/users/dcd9e18442c24cce92cfa75abafad209/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.722355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 802/1599] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.726516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f83b68-68a2-4f15-80c4-7da882809823 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.731809 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcdfe6-d282-452d-ac57-94f35ae7947c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8645ab107c9b4d8bb4015aa8529c6ea5, user_id=dcd9e18442c24cce92cfa75abafad209, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.734553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcdfe6-d282-452d-ac57-94f35ae7947c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.737953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f83b68-68a2-4f15-80c4-7da882809823 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.743975 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26704e-ec19-4213-b643-6382d9feb837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.744567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26704e-ec19-4213-b643-6382d9feb837 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.744892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26704e-ec19-4213-b643-6382d9feb837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.745164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26704e-ec19-4213-b643-6382d9feb837 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.745815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26704e-ec19-4213-b643-6382d9feb837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.747332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 798/1600] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:57 2022] PUT /identity/v3/projects/8645ab107c9b4d8bb4015aa8529c6ea5/users/dcd9e18442c24cce92cfa75abafad209/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:57.747463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26704e-ec19-4213-b643-6382d9feb837 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.748140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcdfe6-d282-452d-ac57-94f35ae7947c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.758215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 803/1601] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.759694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9413e-e791-4adb-b413-d0c72d48533d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.760356 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9413e-e791-4adb-b413-d0c72d48533d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.760648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9413e-e791-4adb-b413-d0c72d48533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.779871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f0e9413e-e791-4adb-b413-d0c72d48533d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:57.781116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f0e9413e-e791-4adb-b413-d0c72d48533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:57.814154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcdfe6-d282-452d-ac57-94f35ae7947c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.818853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5d8ea-d2c5-4a2f-a9f5-9b16d173d0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.819462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5d8ea-d2c5-4a2f-a9f5-9b16d173d0dd admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.819742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5d8ea-d2c5-4a2f-a9f5-9b16d173d0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.820013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5d8ea-d2c5-4a2f-a9f5-9b16d173d0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.820647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5d8ea-d2c5-4a2f-a9f5-9b16d173d0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.823131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5d8ea-d2c5-4a2f-a9f5-9b16d173d0dd admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.833740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 799/1602] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.839237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5d8ea-d2c5-4a2f-a9f5-9b16d173d0dd admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.842044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 804/1603] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:57.842704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9413e-e791-4adb-b413-d0c72d48533d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.849723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5d8ea-d2c5-4a2f-a9f5-9b16d173d0dd admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.852634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9413e-e791-4adb-b413-d0c72d48533d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.854690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab271cb-4315-4fe0-81b3-c22d7624b1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.855351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab271cb-4315-4fe0-81b3-c22d7624b1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.855630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab271cb-4315-4fe0-81b3-c22d7624b1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.855930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab271cb-4315-4fe0-81b3-c22d7624b1d9 admin admin] PATH_INFO: `/v3/projects/055434f8dfd746e9b37f89c67e7ce754/users/4e593c3f01f74f209bc450b14c348d73/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.857359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c743bb6-1ea5-43dd-af12-49ae5a8731c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.858166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c743bb6-1ea5-43dd-af12-49ae5a8731c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.858337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c743bb6-1ea5-43dd-af12-49ae5a8731c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.858595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c743bb6-1ea5-43dd-af12-49ae5a8731c9 admin admin] PATH_INFO: `/v3/projects/1a62540e4d73465c8a17611589370ddd/users/2e97da4529d140358e92175d966d838d/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.871417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab271cb-4315-4fe0-81b3-c22d7624b1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055434f8dfd746e9b37f89c67e7ce754, user_id=4e593c3f01f74f209bc450b14c348d73, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.873392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c743bb6-1ea5-43dd-af12-49ae5a8731c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a62540e4d73465c8a17611589370ddd, user_id=2e97da4529d140358e92175d966d838d, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.874174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab271cb-4315-4fe0-81b3-c22d7624b1d9 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.874432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-1c743bb6-1ea5-43dd-af12-49ae5a8731c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:55:57.874432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 15 10:55:57.874432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: Sep 15 10:55:57.876427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c743bb6-1ea5-43dd-af12-49ae5a8731c9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.890922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 800/1604] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:57 2022] PUT /identity/v3/projects/055434f8dfd746e9b37f89c67e7ce754/users/4e593c3f01f74f209bc450b14c348d73/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:57.890922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb21d5-e675-4124-8669-9601aac9dbcf None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.890922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb21d5-e675-4124-8669-9601aac9dbcf None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.890922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb21d5-e675-4124-8669-9601aac9dbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.891853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 805/1605] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:57 2022] PUT /identity/v3/projects/1a62540e4d73465c8a17611589370ddd/users/2e97da4529d140358e92175d966d838d/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:57.892696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c743bb6-1ea5-43dd-af12-49ae5a8731c9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:57.910118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-25fb21d5-e675-4124-8669-9601aac9dbcf None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:57.911335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-25fb21d5-e675-4124-8669-9601aac9dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:57.920846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:57.953414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:57.969196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c743bb6-1ea5-43dd-af12-49ae5a8731c9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:57.974993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8dafd-f2bb-4cd4-82e3-152569944f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:57.975617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8dafd-f2bb-4cd4-82e3-152569944f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.975902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8dafd-f2bb-4cd4-82e3-152569944f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.976174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8dafd-f2bb-4cd4-82e3-152569944f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.976850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8dafd-f2bb-4cd4-82e3-152569944f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:57.978511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8dafd-f2bb-4cd4-82e3-152569944f0d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:57.982227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 801/1606] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:57.983484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082c76a-8675-47bb-8fd4-0e968c6289fb None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:57.983778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082c76a-8675-47bb-8fd4-0e968c6289fb None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:57.984050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082c76a-8675-47bb-8fd4-0e968c6289fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:57.996759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 806/1607] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:57.996759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8dafd-f2bb-4cd4-82e3-152569944f0d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.003614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3082c76a-8675-47bb-8fd4-0e968c6289fb None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:58.004265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8dafd-f2bb-4cd4-82e3-152569944f0d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.004847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3082c76a-8675-47bb-8fd4-0e968c6289fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:58.009698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fb5e7-bfea-4473-ae50-435b199e41ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.010988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fb5e7-bfea-4473-ae50-435b199e41ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.011276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fb5e7-bfea-4473-ae50-435b199e41ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.011541 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fb5e7-bfea-4473-ae50-435b199e41ce admin admin] PATH_INFO: `/v3/projects/4166728f03e0487f86f210bbcc2ec61a/users/7900daf9ac5d4d119e0411b9681efe90/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.013143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 802/1608] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:58.013899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082c76a-8675-47bb-8fd4-0e968c6289fb None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.023193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082c76a-8675-47bb-8fd4-0e968c6289fb None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.028413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fb5e7-bfea-4473-ae50-435b199e41ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=4166728f03e0487f86f210bbcc2ec61a, user_id=7900daf9ac5d4d119e0411b9681efe90, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.028703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee6d46-0119-4c12-bb10-cc312484e06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.029403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee6d46-0119-4c12-bb10-cc312484e06a admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.029680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee6d46-0119-4c12-bb10-cc312484e06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.029948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee6d46-0119-4c12-bb10-cc312484e06a admin admin] PATH_INFO: `/v3/projects/45cad0c26ac8434da1c3cf7c48aacc39/users/c4139af75eec4ac6a1576f5a40b02e43/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.031554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fb5e7-bfea-4473-ae50-435b199e41ce admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.047276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee6d46-0119-4c12-bb10-cc312484e06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=45cad0c26ac8434da1c3cf7c48aacc39, user_id=c4139af75eec4ac6a1576f5a40b02e43, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.049110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 807/1609] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:57 2022] PUT /identity/v3/projects/4166728f03e0487f86f210bbcc2ec61a/users/7900daf9ac5d4d119e0411b9681efe90/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:58.056199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fb5e7-bfea-4473-ae50-435b199e41ce admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.056318 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee6d46-0119-4c12-bb10-cc312484e06a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.069512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 803/1610] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:58 2022] PUT /identity/v3/projects/45cad0c26ac8434da1c3cf7c48aacc39/users/c4139af75eec4ac6a1576f5a40b02e43/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:58.070423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee6d46-0119-4c12-bb10-cc312484e06a admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.116163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fb5e7-bfea-4473-ae50-435b199e41ce admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.135186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee6d46-0119-4c12-bb10-cc312484e06a admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.139833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5261733-2ce9-4ea5-9677-8e40663d1df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.140441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5261733-2ce9-4ea5-9677-8e40663d1df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.140727 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5261733-2ce9-4ea5-9677-8e40663d1df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.141370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5261733-2ce9-4ea5-9677-8e40663d1df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.141967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5261733-2ce9-4ea5-9677-8e40663d1df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.143659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5261733-2ce9-4ea5-9677-8e40663d1df8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.154722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 804/1611] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:58.156039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40617d75-ec04-4b9a-831d-1c38e8599952 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.156420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40617d75-ec04-4b9a-831d-1c38e8599952 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.156789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40617d75-ec04-4b9a-831d-1c38e8599952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.166923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c99c53-045e-41d7-aaa0-5904a264828d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.167544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c99c53-045e-41d7-aaa0-5904a264828d service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.167832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c99c53-045e-41d7-aaa0-5904a264828d service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.168111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c99c53-045e-41d7-aaa0-5904a264828d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.177327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c99c53-045e-41d7-aaa0-5904a264828d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.178767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c99c53-045e-41d7-aaa0-5904a264828d service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.186414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-40617d75-ec04-4b9a-831d-1c38e8599952 None None] MFA Rules not processed for user `2e97da4529d140358e92175d966d838d`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:58.187608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-40617d75-ec04-4b9a-831d-1c38e8599952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:58.188731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 808/1612] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:55:58.189666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c99c53-045e-41d7-aaa0-5904a264828d service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.198414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c99c53-045e-41d7-aaa0-5904a264828d service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.202375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b20811-bbfc-4b99-a76b-567dae922110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.203147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b20811-bbfc-4b99-a76b-567dae922110 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.203755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b20811-bbfc-4b99-a76b-567dae922110 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.204042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b20811-bbfc-4b99-a76b-567dae922110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.213843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b20811-bbfc-4b99-a76b-567dae922110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.215422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b20811-bbfc-4b99-a76b-567dae922110 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.225523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 809/1613] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:55:58.226545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b20811-bbfc-4b99-a76b-567dae922110 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.236460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b20811-bbfc-4b99-a76b-567dae922110 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.242216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74939863-c566-49ac-8d82-d5909a670ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.243397 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74939863-c566-49ac-8d82-d5909a670ace admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.243836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74939863-c566-49ac-8d82-d5909a670ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.244304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74939863-c566-49ac-8d82-d5909a670ace admin admin] PATH_INFO: `/v3/projects/9bad2083e5cf40c5bf1f853130262e64/users/e62100a950db47738a7f0e3186e2344a/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.260681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74939863-c566-49ac-8d82-d5909a670ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bad2083e5cf40c5bf1f853130262e64, user_id=e62100a950db47738a7f0e3186e2344a, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.263771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74939863-c566-49ac-8d82-d5909a670ace admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.268342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 805/1614] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:58.269243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40617d75-ec04-4b9a-831d-1c38e8599952 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.270603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-40617d75-ec04-4b9a-831d-1c38e8599952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:58.285596 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 810/1615] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:58 2022] PUT /identity/v3/projects/9bad2083e5cf40c5bf1f853130262e64/users/e62100a950db47738a7f0e3186e2344a/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:58.285596 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c09e88-1850-4993-8d2b-a015c63b952b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.285596 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c09e88-1850-4993-8d2b-a015c63b952b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.285596 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c09e88-1850-4993-8d2b-a015c63b952b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.286771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40617d75-ec04-4b9a-831d-1c38e8599952 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.295978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:58.315886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e9c09e88-1850-4993-8d2b-a015c63b952b None None] MFA Rules not processed for user `7900daf9ac5d4d119e0411b9681efe90`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:58.317682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e9c09e88-1850-4993-8d2b-a015c63b952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:58.351672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2752828-7e32-42d5-a7c2-2d1d3492e30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.352894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2752828-7e32-42d5-a7c2-2d1d3492e30b service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.353382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2752828-7e32-42d5-a7c2-2d1d3492e30b service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.353958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2752828-7e32-42d5-a7c2-2d1d3492e30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.356258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:58.409086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 811/1616] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:58.409086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a71a4-7f19-404c-a0bd-a94b239b14e0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.409086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a71a4-7f19-404c-a0bd-a94b239b14e0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.409086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a71a4-7f19-404c-a0bd-a94b239b14e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.435938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2752828-7e32-42d5-a7c2-2d1d3492e30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.437582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-197a71a4-7f19-404c-a0bd-a94b239b14e0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:58.437736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2752828-7e32-42d5-a7c2-2d1d3492e30b service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.438906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-197a71a4-7f19-404c-a0bd-a94b239b14e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:58.453605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 812/1617] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:58.453605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a71a4-7f19-404c-a0bd-a94b239b14e0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.454276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 806/1618] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:55:58.456063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2752828-7e32-42d5-a7c2-2d1d3492e30b service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.462686 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a71a4-7f19-404c-a0bd-a94b239b14e0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.468678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235389b-22ba-4b6d-9f61-39385b45722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.469586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235389b-22ba-4b6d-9f61-39385b45722f admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.470004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235389b-22ba-4b6d-9f61-39385b45722f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.470291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235389b-22ba-4b6d-9f61-39385b45722f admin admin] PATH_INFO: `/v3/projects/055434f8dfd746e9b37f89c67e7ce754/users/4e593c3f01f74f209bc450b14c348d73/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.473126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235389b-22ba-4b6d-9f61-39385b45722f admin admin] RBAC: Authorizing `identity:create_grant(project_id=055434f8dfd746e9b37f89c67e7ce754, user_id=4e593c3f01f74f209bc450b14c348d73, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.473297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2752828-7e32-42d5-a7c2-2d1d3492e30b service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.476849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235389b-22ba-4b6d-9f61-39385b45722f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.479661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fde48-2776-4b8d-a338-a7df2196d13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.480513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fde48-2776-4b8d-a338-a7df2196d13b admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.480900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fde48-2776-4b8d-a338-a7df2196d13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.481177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fde48-2776-4b8d-a338-a7df2196d13b admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.482650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fde48-2776-4b8d-a338-a7df2196d13b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.485835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fde48-2776-4b8d-a338-a7df2196d13b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.500516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 813/1619] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:58 2022] PUT /identity/v3/projects/055434f8dfd746e9b37f89c67e7ce754/users/4e593c3f01f74f209bc450b14c348d73/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:58.500516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6de92b-b845-4331-b52c-e8d4385c4938 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.500516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6de92b-b845-4331-b52c-e8d4385c4938 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.500516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6de92b-b845-4331-b52c-e8d4385c4938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.518207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0e6de92b-b845-4331-b52c-e8d4385c4938 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:58.519466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0e6de92b-b845-4331-b52c-e8d4385c4938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:58.535015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 807/1620] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:55:58 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:58.535015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fde48-2776-4b8d-a338-a7df2196d13b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.585117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 814/1621] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:58.586082 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6de92b-b845-4331-b52c-e8d4385c4938 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.647111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fde48-2776-4b8d-a338-a7df2196d13b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.651719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b57a14-2ad1-4caf-995a-fbdf1010c498 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.652834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b57a14-2ad1-4caf-995a-fbdf1010c498 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.652834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b57a14-2ad1-4caf-995a-fbdf1010c498 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.653162 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b57a14-2ad1-4caf-995a-fbdf1010c498 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.674168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6de92b-b845-4331-b52c-e8d4385c4938 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.731233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:58.739225 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b57a14-2ad1-4caf-995a-fbdf1010c498 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.742034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b57a14-2ad1-4caf-995a-fbdf1010c498 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.747005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41460e-002c-439b-ae1d-f5be2539f870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.748374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41460e-002c-439b-ae1d-f5be2539f870 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.748519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41460e-002c-439b-ae1d-f5be2539f870 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.748965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41460e-002c-439b-ae1d-f5be2539f870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.761608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 808/1622] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:55:58.763326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9c4e8-9fe5-420c-9ed4-abc80082cf60 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.763881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9c4e8-9fe5-420c-9ed4-abc80082cf60 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.764261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9c4e8-9fe5-420c-9ed4-abc80082cf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.814949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-96e9c4e8-9fe5-420c-9ed4-abc80082cf60 None None] MFA Rules not processed for user `4e593c3f01f74f209bc450b14c348d73`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:58.816086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-96e9c4e8-9fe5-420c-9ed4-abc80082cf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:58.819792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:58.922790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41460e-002c-439b-ae1d-f5be2539f870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.923788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 809/1623] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:58.924679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41460e-002c-439b-ae1d-f5be2539f870 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.926019 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9c4e8-9fe5-420c-9ed4-abc80082cf60 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.945613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9c4e8-9fe5-420c-9ed4-abc80082cf60 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:58.954642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac999eb-d112-43ae-97d6-b45276aa6a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:58.955589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac999eb-d112-43ae-97d6-b45276aa6a0d admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:58.956059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac999eb-d112-43ae-97d6-b45276aa6a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:58.956719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac999eb-d112-43ae-97d6-b45276aa6a0d admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:58.957792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac999eb-d112-43ae-97d6-b45276aa6a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:58.960746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac999eb-d112-43ae-97d6-b45276aa6a0d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:58.971416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 815/1624] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:55:58.976164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41460e-002c-439b-ae1d-f5be2539f870 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:58.999440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 810/1625] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:55:58 2022] POST /identity/v3/users => generated 290 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:55:58.999440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac999eb-d112-43ae-97d6-b45276aa6a0d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:59.015069 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac999eb-d112-43ae-97d6-b45276aa6a0d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:59.019629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41460e-002c-439b-ae1d-f5be2539f870 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:59.020830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e5735-45c8-4c70-9d15-9ab6e05c8d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:59.021628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e5735-45c8-4c70-9d15-9ab6e05c8d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:59.022062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e5735-45c8-4c70-9d15-9ab6e05c8d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:59.022425 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e5735-45c8-4c70-9d15-9ab6e05c8d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:59.030545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57967ebd-8c4d-4275-98f3-6e5eafcc0884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:59.031504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57967ebd-8c4d-4275-98f3-6e5eafcc0884 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:59.032012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57967ebd-8c4d-4275-98f3-6e5eafcc0884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:59.032445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57967ebd-8c4d-4275-98f3-6e5eafcc0884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:59.046946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57967ebd-8c4d-4275-98f3-6e5eafcc0884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:59.048971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57967ebd-8c4d-4275-98f3-6e5eafcc0884 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:59.058053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e5735-45c8-4c70-9d15-9ab6e05c8d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:59.059797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e5735-45c8-4c70-9d15-9ab6e05c8d15 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:59.061940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 816/1626] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:55:59.062961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57967ebd-8c4d-4275-98f3-6e5eafcc0884 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:59.076280 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57967ebd-8c4d-4275-98f3-6e5eafcc0884 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:59.080919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 811/1627] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:55:59.081302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64a8a8-aff2-4d03-ac95-57cddd430f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:59.081920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64a8a8-aff2-4d03-ac95-57cddd430f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:59.082206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64a8a8-aff2-4d03-ac95-57cddd430f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:59.082479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64a8a8-aff2-4d03-ac95-57cddd430f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:59.083171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64a8a8-aff2-4d03-ac95-57cddd430f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:59.085019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64a8a8-aff2-4d03-ac95-57cddd430f85 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:59.101387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 817/1628] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:59.101833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e5735-45c8-4c70-9d15-9ab6e05c8d15 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:59.119074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e5735-45c8-4c70-9d15-9ab6e05c8d15 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:59.135426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05864bcc-3be9-43e3-8108-0f7d3befdbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:59.136415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05864bcc-3be9-43e3-8108-0f7d3befdbe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:59.136768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05864bcc-3be9-43e3-8108-0f7d3befdbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:59.137137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05864bcc-3be9-43e3-8108-0f7d3befdbe3 admin admin] PATH_INFO: `/v3/projects/27d3efabbdd54c8ebf551475995805a4/users/ac7a9de9241f446d97be98d331992584/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:59.181698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05864bcc-3be9-43e3-8108-0f7d3befdbe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d3efabbdd54c8ebf551475995805a4, user_id=ac7a9de9241f446d97be98d331992584, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:59.184394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05864bcc-3be9-43e3-8108-0f7d3befdbe3 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:59.212194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 812/1629] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:59 2022] PUT /identity/v3/projects/27d3efabbdd54c8ebf551475995805a4/users/ac7a9de9241f446d97be98d331992584/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:55:59.218257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64a8a8-aff2-4d03-ac95-57cddd430f85 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:59.233919 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64a8a8-aff2-4d03-ac95-57cddd430f85 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:59.256694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05864bcc-3be9-43e3-8108-0f7d3befdbe3 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:59.377422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05864bcc-3be9-43e3-8108-0f7d3befdbe3 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:59.387630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642473-7cce-40a6-9471-4621935d639c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:59.388437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642473-7cce-40a6-9471-4621935d639c service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:59.390226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642473-7cce-40a6-9471-4621935d639c service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:59.390615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642473-7cce-40a6-9471-4621935d639c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:59.394064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f57b0-9874-4c04-afc3-eb23ceebe323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:59.394748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f57b0-9874-4c04-afc3-eb23ceebe323 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:59.395034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f57b0-9874-4c04-afc3-eb23ceebe323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:59.395314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f57b0-9874-4c04-afc3-eb23ceebe323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:59.396022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f57b0-9874-4c04-afc3-eb23ceebe323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:59.401994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f57b0-9874-4c04-afc3-eb23ceebe323 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:59.423284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 818/1630] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:55:59.427967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f57b0-9874-4c04-afc3-eb23ceebe323 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:59.451705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f57b0-9874-4c04-afc3-eb23ceebe323 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:59.462610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156689b-1d7f-4492-acfd-717c25f7a5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:59.463689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156689b-1d7f-4492-acfd-717c25f7a5db admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:59.464101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156689b-1d7f-4492-acfd-717c25f7a5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:59.464500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156689b-1d7f-4492-acfd-717c25f7a5db admin admin] PATH_INFO: `/v3/projects/27d3efabbdd54c8ebf551475995805a4/users/ac7a9de9241f446d97be98d331992584/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:59.466668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156689b-1d7f-4492-acfd-717c25f7a5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d3efabbdd54c8ebf551475995805a4, user_id=ac7a9de9241f446d97be98d331992584, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:59.469660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:55:59.473311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156689b-1d7f-4492-acfd-717c25f7a5db admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:59.492598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 819/1631] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:55:59 2022] PUT /identity/v3/projects/27d3efabbdd54c8ebf551475995805a4/users/ac7a9de9241f446d97be98d331992584/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:55:59.505805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ed535-78c9-498b-aa8b-414ca3ccda3c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:59.506148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ed535-78c9-498b-aa8b-414ca3ccda3c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:59.506871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ed535-78c9-498b-aa8b-414ca3ccda3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:59.516780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22642473-7cce-40a6-9471-4621935d639c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:59.518282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22642473-7cce-40a6-9471-4621935d639c service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:59.540457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 813/1632] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:55:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:55:59.568104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b25ed535-78c9-498b-aa8b-414ca3ccda3c None None] MFA Rules not processed for user `ac7a9de9241f446d97be98d331992584`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:55:59.570589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b25ed535-78c9-498b-aa8b-414ca3ccda3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:55:59.762868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 820/1633] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:55:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:55:59.800223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642473-7cce-40a6-9471-4621935d639c service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:55:59.816639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642473-7cce-40a6-9471-4621935d639c service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:55:59.925865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d1379-e77c-4784-af3b-29857966f0fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:55:59.927092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d1379-e77c-4784-af3b-29857966f0fd service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:55:59.927629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d1379-e77c-4784-af3b-29857966f0fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:55:59.928106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d1379-e77c-4784-af3b-29857966f0fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:55:59.944467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d1379-e77c-4784-af3b-29857966f0fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:55:59.947722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d1379-e77c-4784-af3b-29857966f0fd service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:55:59.982142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 814/1634] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:55:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:00.159728 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ed535-78c9-498b-aa8b-414ca3ccda3c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:00.164040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:00.171984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ed535-78c9-498b-aa8b-414ca3ccda3c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:00.177729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92d19f-351c-4895-9931-ce3e5f78de37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:00.178462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92d19f-351c-4895-9931-ce3e5f78de37 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:00.178787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92d19f-351c-4895-9931-ce3e5f78de37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:00.179112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92d19f-351c-4895-9931-ce3e5f78de37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:00.266747 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92d19f-351c-4895-9931-ce3e5f78de37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:00.268553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92d19f-351c-4895-9931-ce3e5f78de37 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:00.295982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 821/1635] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:02.677012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277c2dd-de91-4065-b078-c66e63f83663 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:02.677012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277c2dd-de91-4065-b078-c66e63f83663 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:02.677012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277c2dd-de91-4065-b078-c66e63f83663 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:02.677012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 815/1636] 10.222.0.50 () {54 vars in 962 bytes} [Thu Sep 15 10:56:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:56:02.688873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d585a9c-4026-4a4b-a3ad-2d6f051e015a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:02.688873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d585a9c-4026-4a4b-a3ad-2d6f051e015a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:02.688873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d585a9c-4026-4a4b-a3ad-2d6f051e015a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:02.735368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2d585a9c-4026-4a4b-a3ad-2d6f051e015a None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:02.735368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2d585a9c-4026-4a4b-a3ad-2d6f051e015a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:02.882382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f510d-c89e-4ef5-9dc1-afcdf48482de None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:02.887724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f510d-c89e-4ef5-9dc1-afcdf48482de None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:02.890089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f510d-c89e-4ef5-9dc1-afcdf48482de None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:02.896913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 816/1638] 10.222.0.50 () {54 vars in 912 bytes} [Thu Sep 15 10:56:02 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:56:02.903495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 822/1638] 10.222.0.50 () {58 vars in 1059 bytes} [Thu Sep 15 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:02.905414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837e7ec-e3d2-4191-bc9c-1b0a3911ecda None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:02.905709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837e7ec-e3d2-4191-bc9c-1b0a3911ecda None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:02.905989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837e7ec-e3d2-4191-bc9c-1b0a3911ecda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:02.948757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-8837e7ec-e3d2-4191-bc9c-1b0a3911ecda None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:02.951376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8837e7ec-e3d2-4191-bc9c-1b0a3911ecda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:02.984949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 817/1639] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:02.994421 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d585a9c-4026-4a4b-a3ad-2d6f051e015a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:03.012770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d585a9c-4026-4a4b-a3ad-2d6f051e015a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:03.017429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88a36a-b0e6-4b8a-a245-d92fb325639b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:03.019217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88a36a-b0e6-4b8a-a245-d92fb325639b service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:03.019617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88a36a-b0e6-4b8a-a245-d92fb325639b service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:03.020087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88a36a-b0e6-4b8a-a245-d92fb325639b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:03.058824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88a36a-b0e6-4b8a-a245-d92fb325639b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:03.060313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88a36a-b0e6-4b8a-a245-d92fb325639b service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:03.075756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 823/1640] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:05.839856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf83f8-ef4d-47b7-9116-fa465c9652de None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:05.840352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf83f8-ef4d-47b7-9116-fa465c9652de None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:05.840439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf83f8-ef4d-47b7-9116-fa465c9652de None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:05.842796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 818/1641] 10.222.0.50 () {54 vars in 962 bytes} [Thu Sep 15 10:56:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:56:05.852152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28be56-a581-4291-ba05-5d8a303de8f3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:05.852930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28be56-a581-4291-ba05-5d8a303de8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:05.852930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28be56-a581-4291-ba05-5d8a303de8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:05.881721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cc28be56-a581-4291-ba05-5d8a303de8f3 None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:05.884380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cc28be56-a581-4291-ba05-5d8a303de8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:05.898010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 824/1642] 10.222.0.50 () {58 vars in 1059 bytes} [Thu Sep 15 10:56:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:05.904026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d36c0-dae6-4202-964c-e4f8b932a5fb None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:05.905209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d36c0-dae6-4202-964c-e4f8b932a5fb None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:05.905728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d36c0-dae6-4202-964c-e4f8b932a5fb None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:05.907467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 819/1643] 10.222.0.50 () {54 vars in 912 bytes} [Thu Sep 15 10:56:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:56:05.912957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbac35-48e9-4fdf-b02c-3fc84cb69f27 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:05.913370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbac35-48e9-4fdf-b02c-3fc84cb69f27 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:05.913768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbac35-48e9-4fdf-b02c-3fc84cb69f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:05.944504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-64fbac35-48e9-4fdf-b02c-3fc84cb69f27 None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:05.946618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-64fbac35-48e9-4fdf-b02c-3fc84cb69f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:05.981607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 825/1644] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 10:56:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:05.989599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d36c0-dae6-4202-964c-e4f8b932a5fb None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:06.007795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d36c0-dae6-4202-964c-e4f8b932a5fb None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:06.013604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3b805-4390-4633-8ebb-cb4f528156de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:06.014668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b805-4390-4633-8ebb-cb4f528156de service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:06.015245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b805-4390-4633-8ebb-cb4f528156de service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:06.015740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b805-4390-4633-8ebb-cb4f528156de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:06.033220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3b805-4390-4633-8ebb-cb4f528156de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:06.043301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3b805-4390-4633-8ebb-cb4f528156de service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:06.077930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 820/1645] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:10.923085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60056dbb-89f5-4a19-aa3a-77a1fd76e2d1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:10.923647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60056dbb-89f5-4a19-aa3a-77a1fd76e2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:10.923843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60056dbb-89f5-4a19-aa3a-77a1fd76e2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:11.006460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-60056dbb-89f5-4a19-aa3a-77a1fd76e2d1 None None] MFA Rules not processed for user `dcd9e18442c24cce92cfa75abafad209`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:11.016040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-60056dbb-89f5-4a19-aa3a-77a1fd76e2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:11.171633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:11.310329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 826/1646] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:11.322858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0a38b-df86-41cc-8328-4db86f741da5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:11.323815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0a38b-df86-41cc-8328-4db86f741da5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:11.324230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0a38b-df86-41cc-8328-4db86f741da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:11.390717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3ab0a38b-df86-41cc-8328-4db86f741da5 None None] MFA Rules not processed for user `dcd9e18442c24cce92cfa75abafad209`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:11.395784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3ab0a38b-df86-41cc-8328-4db86f741da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:11.439120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 821/1647] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:11.460369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60056dbb-89f5-4a19-aa3a-77a1fd76e2d1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:11.485668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60056dbb-89f5-4a19-aa3a-77a1fd76e2d1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:11.504516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9249-78b1-485a-a0e4-dd15c31b9a4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:11.505857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9249-78b1-485a-a0e4-dd15c31b9a4d service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:11.508137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9249-78b1-485a-a0e4-dd15c31b9a4d service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:11.510240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9249-78b1-485a-a0e4-dd15c31b9a4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:11.533554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be9249-78b1-485a-a0e4-dd15c31b9a4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:11.542573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be9249-78b1-485a-a0e4-dd15c31b9a4d service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:11.574934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 827/1648] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:12.558796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd193d-5df5-4d8e-8bbe-3fc5864278a5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:12.559334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd193d-5df5-4d8e-8bbe-3fc5864278a5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:12.559334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd193d-5df5-4d8e-8bbe-3fc5864278a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:12.628954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-6cbd193d-5df5-4d8e-8bbe-3fc5864278a5 None None] MFA Rules not processed for user `e1fdbf0fb6b14090bf38fac073d01814`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:12.630500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6cbd193d-5df5-4d8e-8bbe-3fc5864278a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:12.791083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 822/1649] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:12.800117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9249-78b1-485a-a0e4-dd15c31b9a4d service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:12.819351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9249-78b1-485a-a0e4-dd15c31b9a4d service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:12.837844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf84dc9-8219-4bd7-bc21-9aa077132f14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:12.838634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf84dc9-8219-4bd7-bc21-9aa077132f14 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:12.842255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf84dc9-8219-4bd7-bc21-9aa077132f14 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:12.842765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf84dc9-8219-4bd7-bc21-9aa077132f14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:12.871804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf84dc9-8219-4bd7-bc21-9aa077132f14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:12.873643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf84dc9-8219-4bd7-bc21-9aa077132f14 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:12.891529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 828/1650] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:12.899966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa98a8f-2e73-47e4-9927-f7a3ffee3063 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:12.900444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa98a8f-2e73-47e4-9927-f7a3ffee3063 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:12.900911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa98a8f-2e73-47e4-9927-f7a3ffee3063 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:12.902711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 823/1651] 10.222.0.50 () {54 vars in 912 bytes} [Thu Sep 15 10:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:56:12.909993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f021a7-ced5-4293-a16d-9c730a19fa94 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:12.910509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f021a7-ced5-4293-a16d-9c730a19fa94 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:12.910948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f021a7-ced5-4293-a16d-9c730a19fa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:12.995904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-74f021a7-ced5-4293-a16d-9c730a19fa94 None None] MFA Rules not processed for user `f5e7b08dc3544e76ac6732ca2aa89bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:12.997254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-74f021a7-ced5-4293-a16d-9c730a19fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:13.126153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 829/1652] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 10:56:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:13.156179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa98a8f-2e73-47e4-9927-f7a3ffee3063 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:13.183678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa98a8f-2e73-47e4-9927-f7a3ffee3063 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:13.194297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12146558-6332-4512-94e3-76cad79d4d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:13.195208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12146558-6332-4512-94e3-76cad79d4d69 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:13.195757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12146558-6332-4512-94e3-76cad79d4d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:13.196200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12146558-6332-4512-94e3-76cad79d4d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:13.221074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12146558-6332-4512-94e3-76cad79d4d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:13.221074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12146558-6332-4512-94e3-76cad79d4d69 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:13.249109 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 824/1653] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:13.844077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d3d5-89c4-4226-b9d8-dfe1eeda6457 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:13.844527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d3d5-89c4-4226-b9d8-dfe1eeda6457 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:13.844653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d3d5-89c4-4226-b9d8-dfe1eeda6457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:13.905768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-17e8d3d5-89c4-4226-b9d8-dfe1eeda6457 None None] MFA Rules not processed for user `c4139af75eec4ac6a1576f5a40b02e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:13.907051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-17e8d3d5-89c4-4226-b9d8-dfe1eeda6457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:14.051607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 830/1654] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:14.065120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee585b8-933d-4f4d-8dde-8382f97f50aa None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:14.066729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee585b8-933d-4f4d-8dde-8382f97f50aa None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:14.067208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee585b8-933d-4f4d-8dde-8382f97f50aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:14.111179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5ee585b8-933d-4f4d-8dde-8382f97f50aa None None] MFA Rules not processed for user `c4139af75eec4ac6a1576f5a40b02e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:14.115947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5ee585b8-933d-4f4d-8dde-8382f97f50aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:14.132421 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 825/1655] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:14.150444 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d3d5-89c4-4226-b9d8-dfe1eeda6457 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:14.170267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d3d5-89c4-4226-b9d8-dfe1eeda6457 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:14.184191 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f19205-81f4-4b22-9877-46b7b7377fc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:14.184946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f19205-81f4-4b22-9877-46b7b7377fc7 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:14.185144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f19205-81f4-4b22-9877-46b7b7377fc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:14.185423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f19205-81f4-4b22-9877-46b7b7377fc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:14.207079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f19205-81f4-4b22-9877-46b7b7377fc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:14.208780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f19205-81f4-4b22-9877-46b7b7377fc7 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:14.238655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 831/1656] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:15.010695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee585b8-933d-4f4d-8dde-8382f97f50aa None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:15.084715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:15.221344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee585b8-933d-4f4d-8dde-8382f97f50aa None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:15.238178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da016399-30d6-4fe4-b8b6-6e51434b3e61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:15.239711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da016399-30d6-4fe4-b8b6-6e51434b3e61 service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:15.240203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da016399-30d6-4fe4-b8b6-6e51434b3e61 service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:15.240652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da016399-30d6-4fe4-b8b6-6e51434b3e61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:15.243648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-da016399-30d6-4fe4-b8b6-6e51434b3e61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:15.273304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da016399-30d6-4fe4-b8b6-6e51434b3e61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:15.275477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da016399-30d6-4fe4-b8b6-6e51434b3e61 service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:15.291397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 826/1657] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:15.709788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debd577-158c-470d-9b12-3d764d13c7be None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:15.711208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debd577-158c-470d-9b12-3d764d13c7be None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:15.711635 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debd577-158c-470d-9b12-3d764d13c7be None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:15.713236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 832/1658] 10.222.0.50 () {54 vars in 912 bytes} [Thu Sep 15 10:56:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:56:15.720836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3613f4-dc1c-4eb7-ab47-0161018e59ac None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:15.721376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3613f4-dc1c-4eb7-ab47-0161018e59ac None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:15.721803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3613f4-dc1c-4eb7-ab47-0161018e59ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:15.786387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5a3613f4-dc1c-4eb7-ab47-0161018e59ac None None] MFA Rules not processed for user `f5e7b08dc3544e76ac6732ca2aa89bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:15.787670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5a3613f4-dc1c-4eb7-ab47-0161018e59ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:15.845189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 827/1659] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 10:56:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:15.895838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debd577-158c-470d-9b12-3d764d13c7be None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:15.926839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debd577-158c-470d-9b12-3d764d13c7be None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:15.972301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620641a5-2add-4250-9575-59655b12893c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:15.972977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620641a5-2add-4250-9575-59655b12893c service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:15.973786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620641a5-2add-4250-9575-59655b12893c service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:15.974225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620641a5-2add-4250-9575-59655b12893c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:15.996653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620641a5-2add-4250-9575-59655b12893c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:16.001262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620641a5-2add-4250-9575-59655b12893c service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:16.019816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 833/1660] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:17.361546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e878f6-0912-464b-871f-be1e793bfb66 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:17.362418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e878f6-0912-464b-871f-be1e793bfb66 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:17.362842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e878f6-0912-464b-871f-be1e793bfb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:17.452950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-16e878f6-0912-464b-871f-be1e793bfb66 None None] MFA Rules not processed for user `1eeafdaa1db4436abac229a6f48543a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:17.454202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-16e878f6-0912-464b-871f-be1e793bfb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:17.587179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44c7e2-067d-48d4-afdb-a496fa20d30e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:17.587754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44c7e2-067d-48d4-afdb-a496fa20d30e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:17.587754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44c7e2-067d-48d4-afdb-a496fa20d30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:17.633901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 828/1661] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:17.656377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8f44c7e2-067d-48d4-afdb-a496fa20d30e None None] MFA Rules not processed for user `e62100a950db47738a7f0e3186e2344a`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:17.658839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8f44c7e2-067d-48d4-afdb-a496fa20d30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:17.670674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e878f6-0912-464b-871f-be1e793bfb66 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:17.683015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:17.701134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e878f6-0912-464b-871f-be1e793bfb66 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:17.723213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3e2b4-d723-4246-b693-027b2bdddbba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:17.723842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3e2b4-d723-4246-b693-027b2bdddbba service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:17.724307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3e2b4-d723-4246-b693-027b2bdddbba service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:17.724850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3e2b4-d723-4246-b693-027b2bdddbba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:17.749126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3e2b4-d723-4246-b693-027b2bdddbba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:17.751043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3e2b4-d723-4246-b693-027b2bdddbba service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:17.785993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 829/1662] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:17.918643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 834/1663] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:18.016012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3e2b4-d723-4246-b693-027b2bdddbba service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:18.034044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3e2b4-d723-4246-b693-027b2bdddbba service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:18.041830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b410826-8d33-42eb-81f8-49c3e2f05dbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:18.042735 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b410826-8d33-42eb-81f8-49c3e2f05dbd service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:18.043448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b410826-8d33-42eb-81f8-49c3e2f05dbd service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:18.043943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b410826-8d33-42eb-81f8-49c3e2f05dbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:18.066713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b410826-8d33-42eb-81f8-49c3e2f05dbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:18.069741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b410826-8d33-42eb-81f8-49c3e2f05dbd service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:18.098106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 830/1664] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:18.227870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47431ef-f174-4223-949a-55a98e2f7f92 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:18.228158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47431ef-f174-4223-949a-55a98e2f7f92 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:18.228435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47431ef-f174-4223-949a-55a98e2f7f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:18.284767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f47431ef-f174-4223-949a-55a98e2f7f92 None None] MFA Rules not processed for user `c4139af75eec4ac6a1576f5a40b02e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:18.286295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f47431ef-f174-4223-949a-55a98e2f7f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:18.317881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 835/1665] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:19.266429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b606d13-4cca-4c8d-b3ea-650d999ad81d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:19.272880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b606d13-4cca-4c8d-b3ea-650d999ad81d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:19.274382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b606d13-4cca-4c8d-b3ea-650d999ad81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:19.514262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5b606d13-4cca-4c8d-b3ea-650d999ad81d None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:19.518348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5b606d13-4cca-4c8d-b3ea-650d999ad81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:19.737857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:19.878262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 831/1666] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 10:56:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 614 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:19.959487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47431ef-f174-4223-949a-55a98e2f7f92 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:20.199967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47431ef-f174-4223-949a-55a98e2f7f92 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:20.206510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:20.207546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:20.208070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:20.208573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] PATH_INFO: `/v3/users/20c22261ce094adbad87694311fe0368` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:20.217333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20c22261ce094adbad87694311fe0368)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:20.218844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:56:20.218844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 15 10:56:20.218844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: Sep 15 10:56:20.220941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:20.270227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b606d13-4cca-4c8d-b3ea-650d999ad81d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:20.271994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5b606d13-4cca-4c8d-b3ea-650d999ad81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:20.454386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20c22261ce094adbad87694311fe0368'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:20.486591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20c22261ce094adbad87694311fe0368'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:20.487467 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20c22261ce094adbad87694311fe0368'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:20.532921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20c22261ce094adbad87694311fe0368'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:20.533651 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-760178d9-71c9-419a-bbc1-617de0773478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20c22261ce094adbad87694311fe0368'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:20.603579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b606d13-4cca-4c8d-b3ea-650d999ad81d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:20.605174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 836/1667] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:19 2022] DELETE /identity/v3/users/20c22261ce094adbad87694311fe0368 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:20.609762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517f931-d9bf-4c19-8398-a19d6df46751 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:20.610030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517f931-d9bf-4c19-8398-a19d6df46751 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:20.610308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517f931-d9bf-4c19-8398-a19d6df46751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:20.661916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4517f931-d9bf-4c19-8398-a19d6df46751 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:20.666220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4517f931-d9bf-4c19-8398-a19d6df46751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:20.704624 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:20.761786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:20.762932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:20.765008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:20.766011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:20.770690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:20.877910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 837/1668] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:20.887536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8e1f5-db3d-48a2-bdb0-7e55461393aa None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:20.887821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8e1f5-db3d-48a2-bdb0-7e55461393aa None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:20.890657 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8e1f5-db3d-48a2-bdb0-7e55461393aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:20.988588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-63f8e1f5-db3d-48a2-bdb0-7e55461393aa None None] MFA Rules not processed for user `e1fdbf0fb6b14090bf38fac073d01814`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:20.994541 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-63f8e1f5-db3d-48a2-bdb0-7e55461393aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:21.103240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:21.108190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:21.162939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 832/1669] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 10:56:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 893 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 10:56:21.245248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:21.249790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:21.304196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:21.368666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 838/1670] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:21.570091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf5b38-6027-4f35-95e4-906d0417c956 service swift] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:21.608769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f06e4d-8733-4621-840d-7eeb760af0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:21.613448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f06e4d-8733-4621-840d-7eeb760af0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:21.613826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f06e4d-8733-4621-840d-7eeb760af0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:21.614878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f06e4d-8733-4621-840d-7eeb760af0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:21.678093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f06e4d-8733-4621-840d-7eeb760af0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:21.688961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f06e4d-8733-4621-840d-7eeb760af0b6 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:21.766958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 833/1671] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:23.863995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8e1f5-db3d-48a2-bdb0-7e55461393aa None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:23.865603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-63f8e1f5-db3d-48a2-bdb0-7e55461393aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:23.874073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5c07c-eb97-42f0-ba5a-003c55814efa None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:23.875281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5c07c-eb97-42f0-ba5a-003c55814efa None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:23.875809 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5c07c-eb97-42f0-ba5a-003c55814efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:23.942653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8e1f5-db3d-48a2-bdb0-7e55461393aa None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:23.963501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-bab5c07c-eb97-42f0-ba5a-003c55814efa None None] MFA Rules not processed for user `f5e7b08dc3544e76ac6732ca2aa89bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:23.964202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:23.966206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:23.967932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:23.968709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] PATH_INFO: `/v3/users/7900daf9ac5d4d119e0411b9681efe90` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:23.970991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-bab5c07c-eb97-42f0-ba5a-003c55814efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:23.973630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7900daf9ac5d4d119e0411b9681efe90)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:23.984788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:24.002642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 834/1672] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Sep 15 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:24.125263 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7900daf9ac5d4d119e0411b9681efe90'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:24.139822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7900daf9ac5d4d119e0411b9681efe90'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:24.140726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7900daf9ac5d4d119e0411b9681efe90'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:24.145972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5c07c-eb97-42f0-ba5a-003c55814efa None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:24.164050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7900daf9ac5d4d119e0411b9681efe90'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:24.167416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82e21fce-1e6a-46a4-9092-ee2f6f506292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7900daf9ac5d4d119e0411b9681efe90'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:24.182570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5c07c-eb97-42f0-ba5a-003c55814efa None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:24.196414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:24.200030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:24.200328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:24.200612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:24.205251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:24.229332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 839/1673] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:23 2022] DELETE /identity/v3/users/7900daf9ac5d4d119e0411b9681efe90 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:24.233244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:24.234725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:24.234862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b43bd-631b-4500-934b-0af30972b274 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:24.235047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b43bd-631b-4500-934b-0af30972b274 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:24.235319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b43bd-631b-4500-934b-0af30972b274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:24.254904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 835/1674] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:24.319162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-756b43bd-631b-4500-934b-0af30972b274 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:24.326136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-756b43bd-631b-4500-934b-0af30972b274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:24.463605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:24.593091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 840/1675] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:24.618439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:24.620877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:24.655767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f867245-0b25-4256-8925-c7c068fa93bc service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:24.933366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b68f-472f-4257-ab77-a968bce229d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:24.934118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b68f-472f-4257-ab77-a968bce229d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:24.934469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b68f-472f-4257-ab77-a968bce229d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:24.934813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b68f-472f-4257-ab77-a968bce229d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:24.956587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1b68f-472f-4257-ab77-a968bce229d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:24.960205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1b68f-472f-4257-ab77-a968bce229d1 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:25.000733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 836/1676] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:26.061536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b43bd-631b-4500-934b-0af30972b274 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:26.063717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-756b43bd-631b-4500-934b-0af30972b274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:26.111332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b43bd-631b-4500-934b-0af30972b274 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:26.135137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:26.135866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:26.136182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:26.136474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] PATH_INFO: `/v3/projects/63a6561e6952440da97f5f22dec2799c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:26.138233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a6561e6952440da97f5f22dec2799c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:26.139793 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:26.281489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a6561e6952440da97f5f22dec2799c', 'request_id': 'req-dffb4008-e923-4958-b305-0eaf3dc8ac55'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:26.296698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a6561e6952440da97f5f22dec2799c', 'request_id': 'req-dffb4008-e923-4958-b305-0eaf3dc8ac55'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:26.316861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a6561e6952440da97f5f22dec2799c', 'request_id': 'req-dffb4008-e923-4958-b305-0eaf3dc8ac55'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:26.367617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] The token cache is being invalidate because project 63a6561e6952440da97f5f22dec2799c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:56:26.367953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:26.369218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dffb4008-e923-4958-b305-0eaf3dc8ac55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:26.377001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 841/1677] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:56:26 2022] DELETE /identity/v3/projects/63a6561e6952440da97f5f22dec2799c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:26.379034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b68f-472f-4257-ab77-a968bce229d1 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:26.418918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b68f-472f-4257-ab77-a968bce229d1 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:26.702440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:26.707010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:26.707568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:26.709235 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] PATH_INFO: `/v3/users/4e593c3f01f74f209bc450b14c348d73` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:26.716971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e593c3f01f74f209bc450b14c348d73)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:26.727239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:26.874343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:26.928233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e593c3f01f74f209bc450b14c348d73'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:26.931852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e593c3f01f74f209bc450b14c348d73'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:26.962707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e593c3f01f74f209bc450b14c348d73'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:26.969138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e593c3f01f74f209bc450b14c348d73'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:27.016422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ffbcd604-acc4-425f-aebe-59cee27f6266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e593c3f01f74f209bc450b14c348d73'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:27.130088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 837/1678] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:26 2022] DELETE /identity/v3/users/4e593c3f01f74f209bc450b14c348d73 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:29.328217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f051d-c77d-4df3-9ded-a1d690492c79 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:29.329030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f051d-c77d-4df3-9ded-a1d690492c79 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:29.329620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f051d-c77d-4df3-9ded-a1d690492c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:29.401351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-577f051d-c77d-4df3-9ded-a1d690492c79 None None] MFA Rules not processed for user `e62100a950db47738a7f0e3186e2344a`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:29.405902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-577f051d-c77d-4df3-9ded-a1d690492c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:29.622822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 842/1679] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:29.909107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10abb2f-2610-42b0-8645-f6919f3134b3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:29.910001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10abb2f-2610-42b0-8645-f6919f3134b3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:29.910461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10abb2f-2610-42b0-8645-f6919f3134b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:29.958168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d10abb2f-2610-42b0-8645-f6919f3134b3 None None] MFA Rules not processed for user `1eeafdaa1db4436abac229a6f48543a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:29.959699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d10abb2f-2610-42b0-8645-f6919f3134b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:30.188388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 838/1680] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:30.268653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f051d-c77d-4df3-9ded-a1d690492c79 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:30.270066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-577f051d-c77d-4df3-9ded-a1d690492c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:30.307462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7d40-bfce-4db7-91c6-0077905eba21 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:30.307754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7d40-bfce-4db7-91c6-0077905eba21 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:30.308010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7d40-bfce-4db7-91c6-0077905eba21 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:30.309949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 839/1681] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 10:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:56:30.389776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7d40-bfce-4db7-91c6-0077905eba21 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:30.391524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-571a7d40-bfce-4db7-91c6-0077905eba21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:30.449231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f051d-c77d-4df3-9ded-a1d690492c79 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:30.462418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:30.463070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:30.463505 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:30.463935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] PATH_INFO: `/v3/projects/4166728f03e0487f86f210bbcc2ec61a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:30.468125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4166728f03e0487f86f210bbcc2ec61a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:30.470601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:30.490596 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:30.599956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7d40-bfce-4db7-91c6-0077905eba21 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:30.637764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4166728f03e0487f86f210bbcc2ec61a', 'request_id': 'req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:30.646155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4166728f03e0487f86f210bbcc2ec61a', 'request_id': 'req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:30.666666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4166728f03e0487f86f210bbcc2ec61a', 'request_id': 'req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:30.701853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] The token cache is being invalidate because project 4166728f03e0487f86f210bbcc2ec61a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:56:30.702369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:30.705255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c4b2baeb-cd6c-44ef-8b4e-e3d83cc60e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:30.709285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 843/1682] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:56:30 2022] DELETE /identity/v3/projects/4166728f03e0487f86f210bbcc2ec61a => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:30.721753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb8cd8-be3b-47a9-a222-9c9f046ba7d5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:30.723644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb8cd8-be3b-47a9-a222-9c9f046ba7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:30.724148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb8cd8-be3b-47a9-a222-9c9f046ba7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:30.767843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:30.768498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:30.768772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:30.771596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] PATH_INFO: `/v3/projects/055434f8dfd746e9b37f89c67e7ce754` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:30.773717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=055434f8dfd746e9b37f89c67e7ce754)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:30.774894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: WARNING py.warnings [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 10:56:30.774894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: warnings.warn(msg) Sep 15 10:56:30.774894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: Sep 15 10:56:30.776118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:30.805308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:30.816487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dacb8cd8-be3b-47a9-a222-9c9f046ba7d5 None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:30.817965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dacb8cd8-be3b-47a9-a222-9c9f046ba7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:30.966226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '055434f8dfd746e9b37f89c67e7ce754', 'request_id': 'req-be284c07-da9f-425a-aebe-36278678aa07'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:30.991060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '055434f8dfd746e9b37f89c67e7ce754', 'request_id': 'req-be284c07-da9f-425a-aebe-36278678aa07'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:31.009207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '055434f8dfd746e9b37f89c67e7ce754', 'request_id': 'req-be284c07-da9f-425a-aebe-36278678aa07'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:31.036499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 844/1683] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 10:56:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:31.051912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ec06e-244f-4640-8b32-e9ba2b7a5251 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.052313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ec06e-244f-4640-8b32-e9ba2b7a5251 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.052441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] The token cache is being invalidate because project 055434f8dfd746e9b37f89c67e7ce754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:56:31.052724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ec06e-244f-4640-8b32-e9ba2b7a5251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.053164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:31.056343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-be284c07-da9f-425a-aebe-36278678aa07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:31.062035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 840/1684] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:56:30 2022] DELETE /identity/v3/projects/055434f8dfd746e9b37f89c67e7ce754 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:31.067172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e5f99-e645-40c8-bd43-5e037f737122 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.067603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e5f99-e645-40c8-bd43-5e037f737122 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.067984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e5f99-e645-40c8-bd43-5e037f737122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.104183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e70ec06e-244f-4640-8b32-e9ba2b7a5251 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:31.105627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e70ec06e-244f-4640-8b32-e9ba2b7a5251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:31.110375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-dc4e5f99-e645-40c8-bd43-5e037f737122 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:31.112208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-dc4e5f99-e645-40c8-bd43-5e037f737122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:31.249835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 841/1685] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:31.259573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b20ff-64e5-4d75-9145-1f3493cb1d36 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.259765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b20ff-64e5-4d75-9145-1f3493cb1d36 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.260332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b20ff-64e5-4d75-9145-1f3493cb1d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.273306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 845/1686] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:31.282989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60decef5-f9cd-4657-ae4b-172436902dc5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.283279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60decef5-f9cd-4657-ae4b-172436902dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.283650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60decef5-f9cd-4657-ae4b-172436902dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.313567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-730b20ff-64e5-4d75-9145-1f3493cb1d36 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:31.315110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-730b20ff-64e5-4d75-9145-1f3493cb1d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:31.334216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-60decef5-f9cd-4657-ae4b-172436902dc5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:31.337912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 842/1687] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:31.342070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-60decef5-f9cd-4657-ae4b-172436902dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:31.343152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b20ff-64e5-4d75-9145-1f3493cb1d36 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:31.360089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b20ff-64e5-4d75-9145-1f3493cb1d36 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:31.365159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 846/1688] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:31.370658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60decef5-f9cd-4657-ae4b-172436902dc5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:31.393648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ee6e3-4471-4661-8de7-12ef7305078f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:31.394433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ee6e3-4471-4661-8de7-12ef7305078f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.396432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ee6e3-4471-4661-8de7-12ef7305078f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.396725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ee6e3-4471-4661-8de7-12ef7305078f admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.397901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ee6e3-4471-4661-8de7-12ef7305078f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:56:31.400214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ee6e3-4471-4661-8de7-12ef7305078f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:31.402824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60decef5-f9cd-4657-ae4b-172436902dc5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:31.404042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ee6e3-4471-4661-8de7-12ef7305078f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:31.446407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe213da-b68e-40f0-a103-4dbe34aae202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:31.447456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe213da-b68e-40f0-a103-4dbe34aae202 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.448667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe213da-b68e-40f0-a103-4dbe34aae202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.452313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe213da-b68e-40f0-a103-4dbe34aae202 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.462880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe213da-b68e-40f0-a103-4dbe34aae202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:56:31.467774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 843/1689] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:56:31.468951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe213da-b68e-40f0-a103-4dbe34aae202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:31.470059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ee6e3-4471-4661-8de7-12ef7305078f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:31.480461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe213da-b68e-40f0-a103-4dbe34aae202 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:31.502701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ee6e3-4471-4661-8de7-12ef7305078f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:31.517531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82645bc0-bc44-4258-913a-e35392e0d8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:31.518230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82645bc0-bc44-4258-913a-e35392e0d8bf admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.518501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82645bc0-bc44-4258-913a-e35392e0d8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.518839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82645bc0-bc44-4258-913a-e35392e0d8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.519494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82645bc0-bc44-4258-913a-e35392e0d8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:31.532126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 847/1690] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:56:31.533930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82645bc0-bc44-4258-913a-e35392e0d8bf admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:31.537784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe213da-b68e-40f0-a103-4dbe34aae202 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:31.570779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe213da-b68e-40f0-a103-4dbe34aae202 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:31.584346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8e0ec-edf5-48b9-b5b9-a4d7ea302311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:31.589644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8e0ec-edf5-48b9-b5b9-a4d7ea302311 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.592010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8e0ec-edf5-48b9-b5b9-a4d7ea302311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.592486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8e0ec-edf5-48b9-b5b9-a4d7ea302311 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.593319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8e0ec-edf5-48b9-b5b9-a4d7ea302311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:31.595783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8e0ec-edf5-48b9-b5b9-a4d7ea302311 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:31.605610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:31.617201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 844/1691] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:56:31 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:31.627106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82645bc0-bc44-4258-913a-e35392e0d8bf admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:31.697528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 848/1692] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:56:31 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:31.705259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8e0ec-edf5-48b9-b5b9-a4d7ea302311 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:31.758248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:31.795131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82645bc0-bc44-4258-913a-e35392e0d8bf admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:31.892707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8e0ec-edf5-48b9-b5b9-a4d7ea302311 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:31.908102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e35126-4c05-4d3e-bedb-08332d3e846e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:31.908711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35126-4c05-4d3e-bedb-08332d3e846e admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.909029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35126-4c05-4d3e-bedb-08332d3e846e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.909326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35126-4c05-4d3e-bedb-08332d3e846e admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.910166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e35126-4c05-4d3e-bedb-08332d3e846e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:31.911958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e35126-4c05-4d3e-bedb-08332d3e846e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:31.924100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d254d1-cd43-4acc-bd28-d3ab92a8f1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:31.925213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d254d1-cd43-4acc-bd28-d3ab92a8f1ff admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:31.925693 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d254d1-cd43-4acc-bd28-d3ab92a8f1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:31.926411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d254d1-cd43-4acc-bd28-d3ab92a8f1ff admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:31.927401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d254d1-cd43-4acc-bd28-d3ab92a8f1ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:31.930843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d254d1-cd43-4acc-bd28-d3ab92a8f1ff admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:31.979034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 849/1693] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:56:31 2022] POST /identity/v3/users => generated 291 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:31.981480 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e35126-4c05-4d3e-bedb-08332d3e846e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:31.998937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 845/1694] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:56:31 2022] POST /identity/v3/users => generated 290 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:32.002364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d254d1-cd43-4acc-bd28-d3ab92a8f1ff admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:32.007956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e35126-4c05-4d3e-bedb-08332d3e846e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:32.021875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5b596-7801-4c94-a45d-048b75fa69ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:32.023131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5b596-7801-4c94-a45d-048b75fa69ea admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.023475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5b596-7801-4c94-a45d-048b75fa69ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.023759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5b596-7801-4c94-a45d-048b75fa69ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.028821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5b596-7801-4c94-a45d-048b75fa69ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:32.036335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5b596-7801-4c94-a45d-048b75fa69ea admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:32.041463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d254d1-cd43-4acc-bd28-d3ab92a8f1ff admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:32.061262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448ef2d-91f0-47ef-90bd-f17f1f099afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:32.065494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448ef2d-91f0-47ef-90bd-f17f1f099afe admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.065945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448ef2d-91f0-47ef-90bd-f17f1f099afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.067032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448ef2d-91f0-47ef-90bd-f17f1f099afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.070327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 850/1695] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:32.070847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448ef2d-91f0-47ef-90bd-f17f1f099afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:32.072671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448ef2d-91f0-47ef-90bd-f17f1f099afe admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:32.076642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5b596-7801-4c94-a45d-048b75fa69ea admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:32.109325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5b596-7801-4c94-a45d-048b75fa69ea admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:32.110928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 846/1696] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:32.114230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448ef2d-91f0-47ef-90bd-f17f1f099afe admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:32.124944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073e769-338a-4b4d-9a4a-c7ae254043e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:32.125640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073e769-338a-4b4d-9a4a-c7ae254043e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.125929 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073e769-338a-4b4d-9a4a-c7ae254043e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.126203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073e769-338a-4b4d-9a4a-c7ae254043e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.126857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073e769-338a-4b4d-9a4a-c7ae254043e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:32.135626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073e769-338a-4b4d-9a4a-c7ae254043e5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:32.151469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448ef2d-91f0-47ef-90bd-f17f1f099afe admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:32.169438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ced84-46e7-4fc5-b038-be136c77394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:32.174052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ced84-46e7-4fc5-b038-be136c77394c admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.174859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ced84-46e7-4fc5-b038-be136c77394c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.175359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ced84-46e7-4fc5-b038-be136c77394c admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.175753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 851/1697] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:32.176423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ced84-46e7-4fc5-b038-be136c77394c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:32.177534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073e769-338a-4b4d-9a4a-c7ae254043e5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:32.181186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ced84-46e7-4fc5-b038-be136c77394c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:32.211386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073e769-338a-4b4d-9a4a-c7ae254043e5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:32.223202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 847/1698] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:32.224824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cfabb-3518-49e8-8516-d982a89926bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:32.226389 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cfabb-3518-49e8-8516-d982a89926bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.226683 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cfabb-3518-49e8-8516-d982a89926bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.226957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cfabb-3518-49e8-8516-d982a89926bb admin admin] PATH_INFO: `/v3/projects/13bc3677811243a7ad77266159c1cd12/users/3d3d4ef93b5545c38713eda1b1d0f81e/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.228061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ced84-46e7-4fc5-b038-be136c77394c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:32.259187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ced84-46e7-4fc5-b038-be136c77394c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:32.266832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cfabb-3518-49e8-8516-d982a89926bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bc3677811243a7ad77266159c1cd12, user_id=3d3d4ef93b5545c38713eda1b1d0f81e, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:32.269942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cfabb-3518-49e8-8516-d982a89926bb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:32.271488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b65e1-43e1-4d8f-8814-eb5063140838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:32.273990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b65e1-43e1-4d8f-8814-eb5063140838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.276042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b65e1-43e1-4d8f-8814-eb5063140838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.276832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b65e1-43e1-4d8f-8814-eb5063140838 admin admin] PATH_INFO: `/v3/projects/246a326f5bde4314bd974add2a3d0689/users/640ed253a2a74bd099beea5137980962/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.313702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 852/1699] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:32 2022] PUT /identity/v3/projects/13bc3677811243a7ad77266159c1cd12/users/3d3d4ef93b5545c38713eda1b1d0f81e/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:32.315248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c576-9574-4e70-b685-ceab573000a6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.315663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c576-9574-4e70-b685-ceab573000a6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.315808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c576-9574-4e70-b685-ceab573000a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.350609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b65e1-43e1-4d8f-8814-eb5063140838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246a326f5bde4314bd974add2a3d0689, user_id=640ed253a2a74bd099beea5137980962, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:32.353401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b65e1-43e1-4d8f-8814-eb5063140838 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:32.406168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 848/1700] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:32 2022] PUT /identity/v3/projects/246a326f5bde4314bd974add2a3d0689/users/640ed253a2a74bd099beea5137980962/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:32.408848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-13a9c576-9574-4e70-b685-ceab573000a6 None None] MFA Rules not processed for user `3d3d4ef93b5545c38713eda1b1d0f81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:32.410134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-13a9c576-9574-4e70-b685-ceab573000a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:32.418527 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbc12e-e9ae-40d1-bd91-a2da8ace86c3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.418829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbc12e-e9ae-40d1-bd91-a2da8ace86c3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.419078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbc12e-e9ae-40d1-bd91-a2da8ace86c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.459901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4efbc12e-e9ae-40d1-bd91-a2da8ace86c3 None None] MFA Rules not processed for user `640ed253a2a74bd099beea5137980962`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:32.465996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4efbc12e-e9ae-40d1-bd91-a2da8ace86c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:32.580448 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 853/1701] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:32.584660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9c576-9574-4e70-b685-ceab573000a6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:32.635950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:32.673528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 849/1702] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:32.688136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbc12e-e9ae-40d1-bd91-a2da8ace86c3 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:32.771378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9c576-9574-4e70-b685-ceab573000a6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:32.782130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:32.785732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd78f2d-f1f8-42e0-90cc-d00f542a6939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:32.787905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd78f2d-f1f8-42e0-90cc-d00f542a6939 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.791438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd78f2d-f1f8-42e0-90cc-d00f542a6939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.792250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd78f2d-f1f8-42e0-90cc-d00f542a6939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.795793 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd78f2d-f1f8-42e0-90cc-d00f542a6939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:32.798054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd78f2d-f1f8-42e0-90cc-d00f542a6939 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:32.855772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbc12e-e9ae-40d1-bd91-a2da8ace86c3 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:32.868586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bac16-d96a-4b26-bb2e-99851f8a72d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:32.869260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bac16-d96a-4b26-bb2e-99851f8a72d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:32.869542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bac16-d96a-4b26-bb2e-99851f8a72d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:32.869817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bac16-d96a-4b26-bb2e-99851f8a72d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:32.870604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bac16-d96a-4b26-bb2e-99851f8a72d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:32.872556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bac16-d96a-4b26-bb2e-99851f8a72d0 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:32.879504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 854/1703] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:56:32 2022] POST /identity/v3/projects => generated 377 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:56:32.881025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd78f2d-f1f8-42e0-90cc-d00f542a6939 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:32.933548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 850/1704] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:56:32 2022] POST /identity/v3/projects => generated 379 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:32.938008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bac16-d96a-4b26-bb2e-99851f8a72d0 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:33.124516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bac16-d96a-4b26-bb2e-99851f8a72d0 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:33.138987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7454dc-03b2-4104-8c8d-4648fefe6485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:33.140619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd78f2d-f1f8-42e0-90cc-d00f542a6939 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:33.140982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7454dc-03b2-4104-8c8d-4648fefe6485 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.141266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7454dc-03b2-4104-8c8d-4648fefe6485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.142800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7454dc-03b2-4104-8c8d-4648fefe6485 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:33.143498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7454dc-03b2-4104-8c8d-4648fefe6485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:33.147707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7454dc-03b2-4104-8c8d-4648fefe6485 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:33.156560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbf2be-5e28-4733-acff-8290bc0f5d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:33.157471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbf2be-5e28-4733-acff-8290bc0f5d40 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.157979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbf2be-5e28-4733-acff-8290bc0f5d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.158408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbf2be-5e28-4733-acff-8290bc0f5d40 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:33.159203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbf2be-5e28-4733-acff-8290bc0f5d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:33.176322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbf2be-5e28-4733-acff-8290bc0f5d40 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:33.219586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 851/1705] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:56:32 2022] POST /identity/v3/users => generated 290 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:56:33.223983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7454dc-03b2-4104-8c8d-4648fefe6485 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:33.233957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 855/1706] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:56:32 2022] POST /identity/v3/users => generated 295 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:33.238328 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbf2be-5e28-4733-acff-8290bc0f5d40 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:33.255097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7454dc-03b2-4104-8c8d-4648fefe6485 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:33.275385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbf2be-5e28-4733-acff-8290bc0f5d40 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:33.276635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316b87-0344-4a07-900d-e484dd703d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:33.278878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316b87-0344-4a07-900d-e484dd703d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.280941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316b87-0344-4a07-900d-e484dd703d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.281841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316b87-0344-4a07-900d-e484dd703d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:33.284014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316b87-0344-4a07-900d-e484dd703d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:33.292966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316b87-0344-4a07-900d-e484dd703d1e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:33.297660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac061d-99cb-4d2d-b092-a5202ce3addd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:33.299974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac061d-99cb-4d2d-b092-a5202ce3addd admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.300562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac061d-99cb-4d2d-b092-a5202ce3addd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.302324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac061d-99cb-4d2d-b092-a5202ce3addd admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:33.304386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac061d-99cb-4d2d-b092-a5202ce3addd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:33.311689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac061d-99cb-4d2d-b092-a5202ce3addd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:33.322567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 852/1707] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:56:33.327070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316b87-0344-4a07-900d-e484dd703d1e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:33.339261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 856/1708] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:33.344197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac061d-99cb-4d2d-b092-a5202ce3addd admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:33.371482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316b87-0344-4a07-900d-e484dd703d1e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:33.388657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae02fd4-767b-4d92-a42f-e0d7a09cf3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:33.391037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae02fd4-767b-4d92-a42f-e0d7a09cf3cb admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.392380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac061d-99cb-4d2d-b092-a5202ce3addd admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:33.393358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae02fd4-767b-4d92-a42f-e0d7a09cf3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.393883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae02fd4-767b-4d92-a42f-e0d7a09cf3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:33.395464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae02fd4-767b-4d92-a42f-e0d7a09cf3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:33.400020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae02fd4-767b-4d92-a42f-e0d7a09cf3cb admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:33.408498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497ac7a-27b7-4274-aa26-9e107b9985b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:33.412699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497ac7a-27b7-4274-aa26-9e107b9985b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.414030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497ac7a-27b7-4274-aa26-9e107b9985b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.414610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497ac7a-27b7-4274-aa26-9e107b9985b9 admin admin] PATH_INFO: `/v3/projects/0d0836b81329441c91a9d511dfacd47e/users/0476f776c09f4f29b294e0783ac32d9d/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:33.437538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 853/1709] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:33.440650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae02fd4-767b-4d92-a42f-e0d7a09cf3cb admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:33.471987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497ac7a-27b7-4274-aa26-9e107b9985b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0836b81329441c91a9d511dfacd47e, user_id=0476f776c09f4f29b294e0783ac32d9d, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:33.474839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497ac7a-27b7-4274-aa26-9e107b9985b9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:33.492542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae02fd4-767b-4d92-a42f-e0d7a09cf3cb admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:33.534304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 857/1710] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:33 2022] PUT /identity/v3/projects/0d0836b81329441c91a9d511dfacd47e/users/0476f776c09f4f29b294e0783ac32d9d/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:33.539007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497ac7a-27b7-4274-aa26-9e107b9985b9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:33.664715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe21ee-1ac9-4358-8b15-e825e5455cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:33.665525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe21ee-1ac9-4358-8b15-e825e5455cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.665805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe21ee-1ac9-4358-8b15-e825e5455cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.666090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe21ee-1ac9-4358-8b15-e825e5455cbd admin admin] PATH_INFO: `/v3/projects/ea310b90d318404391b1774241126819/users/d291569ac9824b228fcac404a385b0b1/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:33.708497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe21ee-1ac9-4358-8b15-e825e5455cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea310b90d318404391b1774241126819, user_id=d291569ac9824b228fcac404a385b0b1, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:33.717488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe21ee-1ac9-4358-8b15-e825e5455cbd admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:33.754034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:33.768443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497ac7a-27b7-4274-aa26-9e107b9985b9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:33.779999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 854/1711] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:33 2022] PUT /identity/v3/projects/ea310b90d318404391b1774241126819/users/d291569ac9824b228fcac404a385b0b1/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:33.795638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f4039-9e3a-472d-ab74-957126dbf9aa None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.796133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f4039-9e3a-472d-ab74-957126dbf9aa None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.796739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f4039-9e3a-472d-ab74-957126dbf9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:33.896422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:33.924305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b71a1c-7dc9-4f64-adc1-47ffc8e36f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:33.925291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b71a1c-7dc9-4f64-adc1-47ffc8e36f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.925291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b71a1c-7dc9-4f64-adc1-47ffc8e36f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.926177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b71a1c-7dc9-4f64-adc1-47ffc8e36f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:33.926842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b71a1c-7dc9-4f64-adc1-47ffc8e36f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:33.927886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-374f4039-9e3a-472d-ab74-957126dbf9aa None None] MFA Rules not processed for user `d291569ac9824b228fcac404a385b0b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:33.928471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b71a1c-7dc9-4f64-adc1-47ffc8e36f65 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:33.932915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-374f4039-9e3a-472d-ab74-957126dbf9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:33.961000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 858/1712] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:33.961990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b71a1c-7dc9-4f64-adc1-47ffc8e36f65 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:33.986021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b71a1c-7dc9-4f64-adc1-47ffc8e36f65 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:33.998667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee9bc7-e858-4fbc-bfab-d562b915742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:33.999352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee9bc7-e858-4fbc-bfab-d562b915742f admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:33.999636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee9bc7-e858-4fbc-bfab-d562b915742f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:33.999911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee9bc7-e858-4fbc-bfab-d562b915742f admin admin] PATH_INFO: `/v3/projects/0d0836b81329441c91a9d511dfacd47e/users/0476f776c09f4f29b294e0783ac32d9d/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:34.001957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee9bc7-e858-4fbc-bfab-d562b915742f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0836b81329441c91a9d511dfacd47e, user_id=0476f776c09f4f29b294e0783ac32d9d, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:34.005181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee9bc7-e858-4fbc-bfab-d562b915742f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:34.021638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 859/1713] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:33 2022] PUT /identity/v3/projects/0d0836b81329441c91a9d511dfacd47e/users/0476f776c09f4f29b294e0783ac32d9d/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:34.042185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86979fed-c58e-445a-9177-f6790c7b27a8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:34.043619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86979fed-c58e-445a-9177-f6790c7b27a8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:34.045202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86979fed-c58e-445a-9177-f6790c7b27a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:34.126066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 855/1714] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:34.130462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f4039-9e3a-472d-ab74-957126dbf9aa None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:34.164368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-86979fed-c58e-445a-9177-f6790c7b27a8 None None] MFA Rules not processed for user `0476f776c09f4f29b294e0783ac32d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:34.167159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-86979fed-c58e-445a-9177-f6790c7b27a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:34.188026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f4039-9e3a-472d-ab74-957126dbf9aa None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:34.276955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 860/1715] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:34.297792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86979fed-c58e-445a-9177-f6790c7b27a8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:34.300643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-86979fed-c58e-445a-9177-f6790c7b27a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:34.420454 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d8c86-e610-4d29-8978-dea9e18a4a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:34.421816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d8c86-e610-4d29-8978-dea9e18a4a1e admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:34.422853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d8c86-e610-4d29-8978-dea9e18a4a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:34.423211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d8c86-e610-4d29-8978-dea9e18a4a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:34.432093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86979fed-c58e-445a-9177-f6790c7b27a8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:34.440023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d8c86-e610-4d29-8978-dea9e18a4a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:34.449779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d8c86-e610-4d29-8978-dea9e18a4a1e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:34.458919 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ba4ac-72bd-43a5-acf7-fc1075f75e3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:34.459175 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ba4ac-72bd-43a5-acf7-fc1075f75e3b service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:34.459444 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ba4ac-72bd-43a5-acf7-fc1075f75e3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:34.459736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ba4ac-72bd-43a5-acf7-fc1075f75e3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:34.505080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ba4ac-72bd-43a5-acf7-fc1075f75e3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:34.511819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ba4ac-72bd-43a5-acf7-fc1075f75e3b service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:34.557588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 856/1716] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:56:34 2022] POST /identity/v3/projects => generated 377 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:34.560565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 861/1717] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:56:34.564912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d8c86-e610-4d29-8978-dea9e18a4a1e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:34.831494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d8c86-e610-4d29-8978-dea9e18a4a1e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:34.831682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ba4ac-72bd-43a5-acf7-fc1075f75e3b service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:34.840913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e26e0-1743-4cf2-b1e1-e6a8f38c4742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:34.842921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ba4ac-72bd-43a5-acf7-fc1075f75e3b service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:34.853568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e26e0-1743-4cf2-b1e1-e6a8f38c4742 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:34.854064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e26e0-1743-4cf2-b1e1-e6a8f38c4742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:34.854395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e26e0-1743-4cf2-b1e1-e6a8f38c4742 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:34.855040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e26e0-1743-4cf2-b1e1-e6a8f38c4742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:34.871280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e26e0-1743-4cf2-b1e1-e6a8f38c4742 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:34.875101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:34.952643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 857/1718] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:56:34 2022] POST /identity/v3/users => generated 295 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:34.958380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e26e0-1743-4cf2-b1e1-e6a8f38c4742 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:34.994287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e26e0-1743-4cf2-b1e1-e6a8f38c4742 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:35.014639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6a075-6297-444f-9bee-97f2f4425b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:35.017051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a075-6297-444f-9bee-97f2f4425b21 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:35.018869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a075-6297-444f-9bee-97f2f4425b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:35.019522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a075-6297-444f-9bee-97f2f4425b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:35.047970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bae26-c883-47dc-bb9f-fcdb6453bd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:35.064391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bae26-c883-47dc-bb9f-fcdb6453bd03 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:35.064992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bae26-c883-47dc-bb9f-fcdb6453bd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:35.065350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bae26-c883-47dc-bb9f-fcdb6453bd03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:35.066209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bae26-c883-47dc-bb9f-fcdb6453bd03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:35.075844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bae26-c883-47dc-bb9f-fcdb6453bd03 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:35.153280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 858/1719] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:35.156942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bae26-c883-47dc-bb9f-fcdb6453bd03 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:35.158839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-533bae26-c883-47dc-bb9f-fcdb6453bd03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:35.186710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:35.187079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a075-6297-444f-9bee-97f2f4425b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:35.188666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a075-6297-444f-9bee-97f2f4425b21 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:35.206805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 862/1720] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:35.207714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6a075-6297-444f-9bee-97f2f4425b21 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:35.261538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6a075-6297-444f-9bee-97f2f4425b21 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:35.295264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef032927-a1b2-4f81-823f-d749c82b7008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:35.295963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef032927-a1b2-4f81-823f-d749c82b7008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:35.297416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef032927-a1b2-4f81-823f-d749c82b7008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:35.298285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef032927-a1b2-4f81-823f-d749c82b7008 admin admin] PATH_INFO: `/v3/projects/1591c2c8903e480b9b9625ade049c378/users/6e4de7b6e15d4d88a1b2bc221b819f74/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:35.337164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef032927-a1b2-4f81-823f-d749c82b7008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1591c2c8903e480b9b9625ade049c378, user_id=6e4de7b6e15d4d88a1b2bc221b819f74, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:35.340349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef032927-a1b2-4f81-823f-d749c82b7008 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:35.392410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 863/1721] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:35 2022] PUT /identity/v3/projects/1591c2c8903e480b9b9625ade049c378/users/6e4de7b6e15d4d88a1b2bc221b819f74/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:35.402489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef032927-a1b2-4f81-823f-d749c82b7008 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:35.550753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bae26-c883-47dc-bb9f-fcdb6453bd03 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:35.629532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef032927-a1b2-4f81-823f-d749c82b7008 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:35.635885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae9e62-2252-49ea-8adf-bee2babdd61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:35.636652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae9e62-2252-49ea-8adf-bee2babdd61d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:35.636977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae9e62-2252-49ea-8adf-bee2babdd61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:35.637252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae9e62-2252-49ea-8adf-bee2babdd61d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:35.637903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae9e62-2252-49ea-8adf-bee2babdd61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:35.639667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae9e62-2252-49ea-8adf-bee2babdd61d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:35.662201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 864/1722] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:35.666282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae9e62-2252-49ea-8adf-bee2babdd61d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:35.692382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae9e62-2252-49ea-8adf-bee2babdd61d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:35.703128 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201a9f3-959b-4919-9c41-fa01764def04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:35.703550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4ea24-e58c-446c-a662-e801359c996b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:35.703820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201a9f3-959b-4919-9c41-fa01764def04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:35.704105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201a9f3-959b-4919-9c41-fa01764def04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:35.704437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4ea24-e58c-446c-a662-e801359c996b service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:35.704545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201a9f3-959b-4919-9c41-fa01764def04 admin admin] PATH_INFO: `/v3/projects/1591c2c8903e480b9b9625ade049c378/users/6e4de7b6e15d4d88a1b2bc221b819f74/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:35.704779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4ea24-e58c-446c-a662-e801359c996b service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:35.705090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4ea24-e58c-446c-a662-e801359c996b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:35.707347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201a9f3-959b-4919-9c41-fa01764def04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1591c2c8903e480b9b9625ade049c378, user_id=6e4de7b6e15d4d88a1b2bc221b819f74, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:35.707773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-19c4ea24-e58c-446c-a662-e801359c996b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:35.721171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201a9f3-959b-4919-9c41-fa01764def04 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:35.722005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4ea24-e58c-446c-a662-e801359c996b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:35.723768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4ea24-e58c-446c-a662-e801359c996b service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:35.736591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 859/1723] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:35.759846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 865/1724] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:35 2022] PUT /identity/v3/projects/1591c2c8903e480b9b9625ade049c378/users/6e4de7b6e15d4d88a1b2bc221b819f74/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:56:35.776713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804028a2-1218-4408-b1f4-6268014ccbb6 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:35.777031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804028a2-1218-4408-b1f4-6268014ccbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:35.777381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804028a2-1218-4408-b1f4-6268014ccbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:35.872037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-804028a2-1218-4408-b1f4-6268014ccbb6 None None] MFA Rules not processed for user `6e4de7b6e15d4d88a1b2bc221b819f74`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:35.873641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-804028a2-1218-4408-b1f4-6268014ccbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:36.005716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 860/1725] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:36.033767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201a9f3-959b-4919-9c41-fa01764def04 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:36.079539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:36.115929 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201a9f3-959b-4919-9c41-fa01764def04 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:36.122186 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4e8c4-53b7-45d3-a70c-8c98fe19c6f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:36.122816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4e8c4-53b7-45d3-a70c-8c98fe19c6f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:36.123145 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4e8c4-53b7-45d3-a70c-8c98fe19c6f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:36.123422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4e8c4-53b7-45d3-a70c-8c98fe19c6f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:36.143477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4e8c4-53b7-45d3-a70c-8c98fe19c6f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:36.146079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4e8c4-53b7-45d3-a70c-8c98fe19c6f9 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:36.159693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 866/1726] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:56:36.241803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804028a2-1218-4408-b1f4-6268014ccbb6 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:36.243984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-804028a2-1218-4408-b1f4-6268014ccbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:36.268163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804028a2-1218-4408-b1f4-6268014ccbb6 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:36.274572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729dc0d-9f8c-4716-aa19-9892f463dcdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:36.286973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729dc0d-9f8c-4716-aa19-9892f463dcdf service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:36.287306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729dc0d-9f8c-4716-aa19-9892f463dcdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:36.287751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729dc0d-9f8c-4716-aa19-9892f463dcdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:36.300884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:36.382303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729dc0d-9f8c-4716-aa19-9892f463dcdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:36.383835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729dc0d-9f8c-4716-aa19-9892f463dcdf service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:36.393621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 861/1727] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:56:36.399920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c744eb-6d25-4b49-8e4e-bd8a8ad55923 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:36.400769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c744eb-6d25-4b49-8e4e-bd8a8ad55923 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:36.400769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c744eb-6d25-4b49-8e4e-bd8a8ad55923 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:36.402243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 867/1728] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 10:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:56:36.410149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5e79d-5cb3-4a2f-b3bb-096672aa6923 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:36.410722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5e79d-5cb3-4a2f-b3bb-096672aa6923 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:36.411165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5e79d-5cb3-4a2f-b3bb-096672aa6923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:36.442584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9ad5e79d-5cb3-4a2f-b3bb-096672aa6923 None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:36.444327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9ad5e79d-5cb3-4a2f-b3bb-096672aa6923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:36.537272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 862/1729] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 10:56:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:36.552654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c744eb-6d25-4b49-8e4e-bd8a8ad55923 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:36.554773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-39c744eb-6d25-4b49-8e4e-bd8a8ad55923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:36.577064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c744eb-6d25-4b49-8e4e-bd8a8ad55923 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:36.590455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278c784-3482-4553-91d1-cb1a772fb1b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:36.591076 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278c784-3482-4553-91d1-cb1a772fb1b1 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:36.591430 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278c784-3482-4553-91d1-cb1a772fb1b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:36.591609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278c784-3482-4553-91d1-cb1a772fb1b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:36.602097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278c784-3482-4553-91d1-cb1a772fb1b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:36.603965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278c784-3482-4553-91d1-cb1a772fb1b1 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:36.614749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 868/1730] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:46.521212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=54 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:46.610293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=19 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:51.124059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5e79d-5cb3-4a2f-b3bb-096672aa6923 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:51.237703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5e79d-5cb3-4a2f-b3bb-096672aa6923 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:51.246079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:51.246820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:51.247179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:51.247528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] PATH_INFO: `/v3/users/640ed253a2a74bd099beea5137980962` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:51.248837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=640ed253a2a74bd099beea5137980962)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:51.251139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:51.314254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640ed253a2a74bd099beea5137980962'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:51.314908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640ed253a2a74bd099beea5137980962'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:51.326275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640ed253a2a74bd099beea5137980962'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:51.327980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640ed253a2a74bd099beea5137980962'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:51.337300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640ed253a2a74bd099beea5137980962'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:51.364839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 863/1731] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:51 2022] DELETE /identity/v3/users/640ed253a2a74bd099beea5137980962 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:51.368012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3f77e-743d-4867-b5de-ff3aaf524891 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:51.369038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3f77e-743d-4867-b5de-ff3aaf524891 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:51.369370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3f77e-743d-4867-b5de-ff3aaf524891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:51.391271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7ae3f77e-743d-4867-b5de-ff3aaf524891 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:51.392721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ae3f77e-743d-4867-b5de-ff3aaf524891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:51.476886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 869/1732] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:51.485097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:51.486649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:51.551848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f093303-00d2-412c-ade0-c6ea348be569 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:51.556456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4ffae-ddd4-4597-bf5f-f4a3ab21fad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:51.557110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4ffae-ddd4-4597-bf5f-f4a3ab21fad1 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:51.557388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4ffae-ddd4-4597-bf5f-f4a3ab21fad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:51.557686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4ffae-ddd4-4597-bf5f-f4a3ab21fad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:51.569215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4ffae-ddd4-4597-bf5f-f4a3ab21fad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:51.570695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4ffae-ddd4-4597-bf5f-f4a3ab21fad1 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:51.582432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 864/1733] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:52.102045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3f77e-743d-4867-b5de-ff3aaf524891 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:52.103593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ae3f77e-743d-4867-b5de-ff3aaf524891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:52.116652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3f77e-743d-4867-b5de-ff3aaf524891 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:52.121387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:52.122027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:52.122308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:52.122582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] PATH_INFO: `/v3/projects/246a326f5bde4314bd974add2a3d0689` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:52.123808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=246a326f5bde4314bd974add2a3d0689)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:52.126633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:52.172932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246a326f5bde4314bd974add2a3d0689', 'request_id': 'req-93042220-6683-4c9d-abf8-d53494bb7caf'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.179299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246a326f5bde4314bd974add2a3d0689', 'request_id': 'req-93042220-6683-4c9d-abf8-d53494bb7caf'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.186911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246a326f5bde4314bd974add2a3d0689', 'request_id': 'req-93042220-6683-4c9d-abf8-d53494bb7caf'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.200237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] The token cache is being invalidate because project 246a326f5bde4314bd974add2a3d0689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:56:52.200525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.201171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.202669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 870/1734] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:56:52 2022] DELETE /identity/v3/projects/246a326f5bde4314bd974add2a3d0689 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:56:52.206616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4ffae-ddd4-4597-bf5f-f4a3ab21fad1 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:52.217029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4ffae-ddd4-4597-bf5f-f4a3ab21fad1 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:52.235640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:52.277942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:52.278576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:52.278855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:52.279123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] PATH_INFO: `/v3/users/d291569ac9824b228fcac404a385b0b1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:52.280279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d291569ac9824b228fcac404a385b0b1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:52.282310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:52.332847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd291569ac9824b228fcac404a385b0b1'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.333832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd291569ac9824b228fcac404a385b0b1'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.343595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd291569ac9824b228fcac404a385b0b1'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.344408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd291569ac9824b228fcac404a385b0b1'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.352407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd291569ac9824b228fcac404a385b0b1'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:52.369778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 865/1735] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:52 2022] DELETE /identity/v3/users/d291569ac9824b228fcac404a385b0b1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:52.857717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:52.860475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:52.904945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93042220-6683-4c9d-abf8-d53494bb7caf admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:52.975333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:52.975983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:52.976258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:52.976526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] PATH_INFO: `/v3/projects/ea310b90d318404391b1774241126819` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:52.977862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea310b90d318404391b1774241126819)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:52.980538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:53.015828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:53.034914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea310b90d318404391b1774241126819', 'request_id': 'req-7f7eb53c-e96d-4c02-86aa-4da21f50057f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.041226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea310b90d318404391b1774241126819', 'request_id': 'req-7f7eb53c-e96d-4c02-86aa-4da21f50057f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.049592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea310b90d318404391b1774241126819', 'request_id': 'req-7f7eb53c-e96d-4c02-86aa-4da21f50057f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.062094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] The token cache is being invalidate because project ea310b90d318404391b1774241126819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:56:53.062537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.063229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.067462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 871/1736] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:56:52 2022] DELETE /identity/v3/projects/ea310b90d318404391b1774241126819 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:56:53.069243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:53.070955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:53.087614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8558f-f1e0-4cd1-a951-5c0de9e6674b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:53.149511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:53.150194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:53.150476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:53.150751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] PATH_INFO: `/v3/users/6e4de7b6e15d4d88a1b2bc221b819f74` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:53.151919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4de7b6e15d4d88a1b2bc221b819f74)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:53.153897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:53.202179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:53.205048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4de7b6e15d4d88a1b2bc221b819f74'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.206280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4de7b6e15d4d88a1b2bc221b819f74'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.215050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4de7b6e15d4d88a1b2bc221b819f74'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.216012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4de7b6e15d4d88a1b2bc221b819f74'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.223011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-460e65a2-aa79-420a-92e2-a5c942f591c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4de7b6e15d4d88a1b2bc221b819f74'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:53.243448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 866/1737] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:53 2022] DELETE /identity/v3/users/6e4de7b6e15d4d88a1b2bc221b819f74 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:56:54.055996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:54.059936 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:54.103989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7eb53c-e96d-4c02-86aa-4da21f50057f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:54.252794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:54.253694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:54.254158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:54.254629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] PATH_INFO: `/v3/users/3d3d4ef93b5545c38713eda1b1d0f81e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:54.257124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3d4ef93b5545c38713eda1b1d0f81e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:54.262273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:54.353372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3d4ef93b5545c38713eda1b1d0f81e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:54.376924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3d4ef93b5545c38713eda1b1d0f81e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:54.377669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3d4ef93b5545c38713eda1b1d0f81e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:54.391906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3d4ef93b5545c38713eda1b1d0f81e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:54.392781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3d4ef93b5545c38713eda1b1d0f81e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:54.438235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 872/1738] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:54 2022] DELETE /identity/v3/users/3d3d4ef93b5545c38713eda1b1d0f81e => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:54.442887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a1e54-f087-4470-8e2c-5b22ba2faa82 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:54.443546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a1e54-f087-4470-8e2c-5b22ba2faa82 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:54.444748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a1e54-f087-4470-8e2c-5b22ba2faa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:54.493151 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-cb7a1e54-f087-4470-8e2c-5b22ba2faa82 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:54.494720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cb7a1e54-f087-4470-8e2c-5b22ba2faa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:54.725763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 867/1739] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:54.823272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:54.827377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:54.936637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:54.963461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ddbb8-ce60-49bf-851d-4968b372ce93 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:54.964029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ddbb8-ce60-49bf-851d-4968b372ce93 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:54.964499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ddbb8-ce60-49bf-851d-4968b372ce93 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:54.965940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 868/1740] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 10:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 10:56:54.979407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d881f-3e2b-4ec1-9728-f0ce0a8f1df8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:54.985003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948bb07-e871-441b-81fc-441eed3e8f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:54.985410 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e795b-8dd8-43d5-9e7a-629a669493f7 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:54.985877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e795b-8dd8-43d5-9e7a-629a669493f7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:54.986145 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948bb07-e871-441b-81fc-441eed3e8f67 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:54.986503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948bb07-e871-441b-81fc-441eed3e8f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:54.986717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e795b-8dd8-43d5-9e7a-629a669493f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:54.987511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948bb07-e871-441b-81fc-441eed3e8f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:55.003817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948bb07-e871-441b-81fc-441eed3e8f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:55.007095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948bb07-e871-441b-81fc-441eed3e8f67 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:55.024049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9d9e795b-8dd8-43d5-9e7a-629a669493f7 None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:55.025891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9d9e795b-8dd8-43d5-9e7a-629a669493f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:55.058425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 873/1741] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:55.143012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 869/1742] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:55.168094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948bb07-e871-441b-81fc-441eed3e8f67 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:55.170223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6948bb07-e871-441b-81fc-441eed3e8f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:55.223033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948bb07-e871-441b-81fc-441eed3e8f67 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:55.236237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3dc8-9176-4dda-941f-da3da3e41519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:55.237101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3dc8-9176-4dda-941f-da3da3e41519 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:55.237391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3dc8-9176-4dda-941f-da3da3e41519 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:55.238134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3dc8-9176-4dda-941f-da3da3e41519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:55.251572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf3dc8-9176-4dda-941f-da3da3e41519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:55.253548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf3dc8-9176-4dda-941f-da3da3e41519 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:55.277595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 874/1743] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:56:55.596185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e795b-8dd8-43d5-9e7a-629a669493f7 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:55.597753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9d9e795b-8dd8-43d5-9e7a-629a669493f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:55.614096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e795b-8dd8-43d5-9e7a-629a669493f7 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:55.620445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:55.621370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:55.621814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:55.622249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] PATH_INFO: `/v3/projects/1591c2c8903e480b9b9625ade049c378` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:55.623561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1591c2c8903e480b9b9625ade049c378)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:55.625682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:55.717059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:55.719410 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1591c2c8903e480b9b9625ade049c378', 'request_id': 'req-c234f227-c173-480c-9c0b-eacf3e641396'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:55.730204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1591c2c8903e480b9b9625ade049c378', 'request_id': 'req-c234f227-c173-480c-9c0b-eacf3e641396'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:55.738741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3dc8-9176-4dda-941f-da3da3e41519 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:55.740411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ecbf3dc8-9176-4dda-941f-da3da3e41519 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:55.764278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3dc8-9176-4dda-941f-da3da3e41519 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:55.768468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1591c2c8903e480b9b9625ade049c378', 'request_id': 'req-c234f227-c173-480c-9c0b-eacf3e641396'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:55.798656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] The token cache is being invalidate because project 1591c2c8903e480b9b9625ade049c378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:56:55.798969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:55.800895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c234f227-c173-480c-9c0b-eacf3e641396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:55.806168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 870/1744] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:56:55 2022] DELETE /identity/v3/projects/1591c2c8903e480b9b9625ade049c378 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:55.813338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bdb54-c40d-4ec7-b702-dd18f15dca9b None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:55.813632 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bdb54-c40d-4ec7-b702-dd18f15dca9b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:55.813913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bdb54-c40d-4ec7-b702-dd18f15dca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:55.874310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e44bdb54-c40d-4ec7-b702-dd18f15dca9b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:55.875767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e44bdb54-c40d-4ec7-b702-dd18f15dca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:55.932330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:55.933032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:55.933640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:55.934140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] PATH_INFO: `/v3/projects/13bc3677811243a7ad77266159c1cd12` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:55.935887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13bc3677811243a7ad77266159c1cd12)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:55.939480 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:56.012564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 871/1745] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:56.021282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06260a3c-ed3f-4ba4-b7d9-d5215d0894c2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:56.022940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06260a3c-ed3f-4ba4-b7d9-d5215d0894c2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:56.024440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06260a3c-ed3f-4ba4-b7d9-d5215d0894c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:56.033211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13bc3677811243a7ad77266159c1cd12', 'request_id': 'req-853361b6-693d-4772-b3c8-c6c8357f0827'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.043417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13bc3677811243a7ad77266159c1cd12', 'request_id': 'req-853361b6-693d-4772-b3c8-c6c8357f0827'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.057108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13bc3677811243a7ad77266159c1cd12', 'request_id': 'req-853361b6-693d-4772-b3c8-c6c8357f0827'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.069509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-06260a3c-ed3f-4ba4-b7d9-d5215d0894c2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:56.071101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-06260a3c-ed3f-4ba4-b7d9-d5215d0894c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:56.092219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] The token cache is being invalidate because project 13bc3677811243a7ad77266159c1cd12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:56:56.092766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.093916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.113129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 875/1746] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:56:55 2022] DELETE /identity/v3/projects/13bc3677811243a7ad77266159c1cd12 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:56.113536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:56.115537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:56.147079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 872/1747] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:56.148116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06260a3c-ed3f-4ba4-b7d9-d5215d0894c2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:56.161054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06260a3c-ed3f-4ba4-b7d9-d5215d0894c2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:56.168623 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0b4d8-251f-482e-b8ae-7a89f53443fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:56.169266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0b4d8-251f-482e-b8ae-7a89f53443fb admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:56.169621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0b4d8-251f-482e-b8ae-7a89f53443fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:56.170245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0b4d8-251f-482e-b8ae-7a89f53443fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:56.170996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0b4d8-251f-482e-b8ae-7a89f53443fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:56:56.171382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0b4d8-251f-482e-b8ae-7a89f53443fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:56.173220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0b4d8-251f-482e-b8ae-7a89f53443fb admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:56.193004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 873/1748] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:56:56.199745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0b4d8-251f-482e-b8ae-7a89f53443fb admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:56.213211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0b4d8-251f-482e-b8ae-7a89f53443fb admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:56.219850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7720ed1-03bc-4d68-b141-d3acbc3b1af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:56.220630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7720ed1-03bc-4d68-b141-d3acbc3b1af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:56.221041 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7720ed1-03bc-4d68-b141-d3acbc3b1af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:56.221433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7720ed1-03bc-4d68-b141-d3acbc3b1af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:56.223128 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7720ed1-03bc-4d68-b141-d3acbc3b1af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:56.225888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7720ed1-03bc-4d68-b141-d3acbc3b1af6 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:56.233024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:56.257680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853361b6-693d-4772-b3c8-c6c8357f0827 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:56.266802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:56.267748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:56.269214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:56.270228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] PATH_INFO: `/v3/users/0476f776c09f4f29b294e0783ac32d9d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:56.272503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0476f776c09f4f29b294e0783ac32d9d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:56.275218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:56.283267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 874/1749] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:56:56 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:56.288654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7720ed1-03bc-4d68-b141-d3acbc3b1af6 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:56.370819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:56.376198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0476f776c09f4f29b294e0783ac32d9d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.393125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0476f776c09f4f29b294e0783ac32d9d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.394997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0476f776c09f4f29b294e0783ac32d9d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.409254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0476f776c09f4f29b294e0783ac32d9d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.410093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7720ed1-03bc-4d68-b141-d3acbc3b1af6 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:56.413018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0476f776c09f4f29b294e0783ac32d9d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:56.423066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e9388-404b-40a3-b652-b23799bd9544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:56.425280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e9388-404b-40a3-b652-b23799bd9544 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:56.427215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e9388-404b-40a3-b652-b23799bd9544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:56.428352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e9388-404b-40a3-b652-b23799bd9544 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:56.431059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e9388-404b-40a3-b652-b23799bd9544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:56.436409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e9388-404b-40a3-b652-b23799bd9544 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:56.452654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 876/1750] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:56 2022] DELETE /identity/v3/users/0476f776c09f4f29b294e0783ac32d9d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:56:56.495085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 875/1751] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:56:56 2022] POST /identity/v3/users => generated 291 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:56.500617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:56.502664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:56.627948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f412a63-a34c-44ba-8366-b4050f7a0e64 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:56.635978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0e6e4-0a7e-4955-a732-077c262d6e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:56.636965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0e6e4-0a7e-4955-a732-077c262d6e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:56.637352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0e6e4-0a7e-4955-a732-077c262d6e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:56.638714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0e6e4-0a7e-4955-a732-077c262d6e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:56.639721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0e6e4-0a7e-4955-a732-077c262d6e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:56.642125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0e6e4-0a7e-4955-a732-077c262d6e59 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:56.659834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 877/1752] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:56.662501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e9388-404b-40a3-b652-b23799bd9544 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:56.682098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e9388-404b-40a3-b652-b23799bd9544 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:56.697345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fe9e0-e1cc-442f-8e7b-33e8e6977e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:56.698613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fe9e0-e1cc-442f-8e7b-33e8e6977e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:56.698740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fe9e0-e1cc-442f-8e7b-33e8e6977e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:56.699254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fe9e0-e1cc-442f-8e7b-33e8e6977e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:56.699926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fe9e0-e1cc-442f-8e7b-33e8e6977e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:56.701580 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fe9e0-e1cc-442f-8e7b-33e8e6977e6d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:56.729597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 876/1753] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:56.730674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0e6e4-0a7e-4955-a732-077c262d6e59 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:56.746635 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0e6e4-0a7e-4955-a732-077c262d6e59 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:56.756117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b58f27-a819-459d-8a29-c501516f0fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:56.757537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b58f27-a819-459d-8a29-c501516f0fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:56.757988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b58f27-a819-459d-8a29-c501516f0fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:56.759001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b58f27-a819-459d-8a29-c501516f0fa1 admin admin] PATH_INFO: `/v3/projects/57194ea9a3544c0eb4eeac25f804a93d/users/b9660451b42142aaad441e9966f1276c/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:56.787671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b58f27-a819-459d-8a29-c501516f0fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57194ea9a3544c0eb4eeac25f804a93d, user_id=b9660451b42142aaad441e9966f1276c, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:56.792994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b58f27-a819-459d-8a29-c501516f0fa1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:56.836738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 878/1754] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:56 2022] PUT /identity/v3/projects/57194ea9a3544c0eb4eeac25f804a93d/users/b9660451b42142aaad441e9966f1276c/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:56.837014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168c3d-9495-4f80-bed9-849a8dd1608b None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:56.837014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168c3d-9495-4f80-bed9-849a8dd1608b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:56.837014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168c3d-9495-4f80-bed9-849a8dd1608b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:56.883412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-09168c3d-9495-4f80-bed9-849a8dd1608b None None] MFA Rules not processed for user `b9660451b42142aaad441e9966f1276c`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:56.884998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-09168c3d-9495-4f80-bed9-849a8dd1608b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:56.985038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 877/1755] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:56.989341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b58f27-a819-459d-8a29-c501516f0fa1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.071717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:57.106969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b58f27-a819-459d-8a29-c501516f0fa1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.113392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31151e5-8ee6-4f04-9f8b-47433156cde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.114242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31151e5-8ee6-4f04-9f8b-47433156cde9 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.114700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31151e5-8ee6-4f04-9f8b-47433156cde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.115475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31151e5-8ee6-4f04-9f8b-47433156cde9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.116292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31151e5-8ee6-4f04-9f8b-47433156cde9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.118457 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31151e5-8ee6-4f04-9f8b-47433156cde9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.156836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 879/1756] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:56:56 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:57.160689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09168c3d-9495-4f80-bed9-849a8dd1608b None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.179298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09168c3d-9495-4f80-bed9-849a8dd1608b None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.240491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019379b-3b26-4be5-9b5b-864a112724a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.241135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019379b-3b26-4be5-9b5b-864a112724a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.241413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019379b-3b26-4be5-9b5b-864a112724a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.241679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019379b-3b26-4be5-9b5b-864a112724a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.242316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019379b-3b26-4be5-9b5b-864a112724a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.244248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019379b-3b26-4be5-9b5b-864a112724a6 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.269139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 878/1757] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:56:57 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:56:57.270201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31151e5-8ee6-4f04-9f8b-47433156cde9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.274181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019379b-3b26-4be5-9b5b-864a112724a6 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.276432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d019379b-3b26-4be5-9b5b-864a112724a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:57.282468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31151e5-8ee6-4f04-9f8b-47433156cde9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.288341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776345a-8f9f-4fc4-b38e-b51c45bdb2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.289020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776345a-8f9f-4fc4-b38e-b51c45bdb2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.289302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776345a-8f9f-4fc4-b38e-b51c45bdb2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.289570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776345a-8f9f-4fc4-b38e-b51c45bdb2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.290481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019379b-3b26-4be5-9b5b-864a112724a6 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.290602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776345a-8f9f-4fc4-b38e-b51c45bdb2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.294765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.295402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.295686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.295957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] PATH_INFO: `/v3/projects/0d0836b81329441c91a9d511dfacd47e` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.296698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776345a-8f9f-4fc4-b38e-b51c45bdb2c0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.297100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0836b81329441c91a9d511dfacd47e)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.299005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.312320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 880/1758] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:57.313052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776345a-8f9f-4fc4-b38e-b51c45bdb2c0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.323282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776345a-8f9f-4fc4-b38e-b51c45bdb2c0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.328240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83191d-7bc4-459a-80a4-b1fa751cf061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.328919 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83191d-7bc4-459a-80a4-b1fa751cf061 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.329200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83191d-7bc4-459a-80a4-b1fa751cf061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.329470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83191d-7bc4-459a-80a4-b1fa751cf061 admin admin] PATH_INFO: `/v3/projects/ac66ac23cde24c10b2b924afba0d58ff/users/4864f18b63ee4deaae44fbe24323f4e7/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.343279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:57.347383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83191d-7bc4-459a-80a4-b1fa751cf061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac66ac23cde24c10b2b924afba0d58ff, user_id=4864f18b63ee4deaae44fbe24323f4e7, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.351420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83191d-7bc4-459a-80a4-b1fa751cf061 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.357047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0836b81329441c91a9d511dfacd47e', 'request_id': 'req-65a58b5c-a363-4b84-8a54-2f9d25179d18'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:57.365327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0836b81329441c91a9d511dfacd47e', 'request_id': 'req-65a58b5c-a363-4b84-8a54-2f9d25179d18'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:57.373100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 881/1759] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:57 2022] PUT /identity/v3/projects/ac66ac23cde24c10b2b924afba0d58ff/users/4864f18b63ee4deaae44fbe24323f4e7/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:57.374663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0836b81329441c91a9d511dfacd47e', 'request_id': 'req-65a58b5c-a363-4b84-8a54-2f9d25179d18'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:57.377931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83191d-7bc4-459a-80a4-b1fa751cf061 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.391259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] The token cache is being invalidate because project 0d0836b81329441c91a9d511dfacd47e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:56:57.391552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:57.392213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-65a58b5c-a363-4b84-8a54-2f9d25179d18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:57.393967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 879/1760] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:56:57 2022] DELETE /identity/v3/projects/0d0836b81329441c91a9d511dfacd47e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:56:57.399427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937325a-536a-4b7e-bbf1-e8659d852df3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.399710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937325a-536a-4b7e-bbf1-e8659d852df3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.399985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937325a-536a-4b7e-bbf1-e8659d852df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.419703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-a937325a-536a-4b7e-bbf1-e8659d852df3 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:57.421332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a937325a-536a-4b7e-bbf1-e8659d852df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:57.442755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83191d-7bc4-459a-80a4-b1fa751cf061 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.448245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8663a-2e73-47dd-8de6-4d0e080d1592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.448879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8663a-2e73-47dd-8de6-4d0e080d1592 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.449413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8663a-2e73-47dd-8de6-4d0e080d1592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.449732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8663a-2e73-47dd-8de6-4d0e080d1592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.450368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8663a-2e73-47dd-8de6-4d0e080d1592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.451967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8663a-2e73-47dd-8de6-4d0e080d1592 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.464277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 882/1761] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:56:57.468044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8663a-2e73-47dd-8de6-4d0e080d1592 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.469822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a4c8663a-2e73-47dd-8de6-4d0e080d1592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:57.487142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8663a-2e73-47dd-8de6-4d0e080d1592 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.491985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 880/1762] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:57.501671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6cdca-03e4-4109-a20b-2dcb86418e46 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.502034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6cdca-03e4-4109-a20b-2dcb86418e46 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.502490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6cdca-03e4-4109-a20b-2dcb86418e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.502891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe054b2-3527-4c20-86eb-144a2a068681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.503800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe054b2-3527-4c20-86eb-144a2a068681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.504256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe054b2-3527-4c20-86eb-144a2a068681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.504722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe054b2-3527-4c20-86eb-144a2a068681 admin admin] PATH_INFO: `/v3/projects/ac66ac23cde24c10b2b924afba0d58ff/users/4864f18b63ee4deaae44fbe24323f4e7/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.509060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe054b2-3527-4c20-86eb-144a2a068681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac66ac23cde24c10b2b924afba0d58ff, user_id=4864f18b63ee4deaae44fbe24323f4e7, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.512051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe054b2-3527-4c20-86eb-144a2a068681 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.525387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-89b6cdca-03e4-4109-a20b-2dcb86418e46 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:57.527301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-89b6cdca-03e4-4109-a20b-2dcb86418e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:57.534553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 883/1763] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:57 2022] PUT /identity/v3/projects/ac66ac23cde24c10b2b924afba0d58ff/users/4864f18b63ee4deaae44fbe24323f4e7/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:57.541038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c2090-2142-4590-ba1f-136409868c3c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.541038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c2090-2142-4590-ba1f-136409868c3c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.541038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c2090-2142-4590-ba1f-136409868c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.579900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6f8c2090-2142-4590-ba1f-136409868c3c None None] MFA Rules not processed for user `4864f18b63ee4deaae44fbe24323f4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:57.582030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6f8c2090-2142-4590-ba1f-136409868c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:57.637355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 881/1764] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:57.638287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6cdca-03e4-4109-a20b-2dcb86418e46 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.650322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6cdca-03e4-4109-a20b-2dcb86418e46 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.655535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d0b62-a92f-40af-af8d-9a0bcd913086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.656144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d0b62-a92f-40af-af8d-9a0bcd913086 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.656432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d0b62-a92f-40af-af8d-9a0bcd913086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.657019 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d0b62-a92f-40af-af8d-9a0bcd913086 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.658331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d0b62-a92f-40af-af8d-9a0bcd913086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:56:57.658645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d0b62-a92f-40af-af8d-9a0bcd913086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.660264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d0b62-a92f-40af-af8d-9a0bcd913086 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.669849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 884/1765] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:57.676005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 882/1766] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:56:57.680895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c2090-2142-4590-ba1f-136409868c3c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.681118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d0b62-a92f-40af-af8d-9a0bcd913086 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.682950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6f8c2090-2142-4590-ba1f-136409868c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:57.690989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:57.692238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d0b62-a92f-40af-af8d-9a0bcd913086 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.700504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6e819-b8f8-4122-b751-a2bd15c6be14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.702054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6e819-b8f8-4122-b751-a2bd15c6be14 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.702054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6e819-b8f8-4122-b751-a2bd15c6be14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.702789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6e819-b8f8-4122-b751-a2bd15c6be14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.703284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6e819-b8f8-4122-b751-a2bd15c6be14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.706441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6e819-b8f8-4122-b751-a2bd15c6be14 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.752035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 883/1767] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:56:57 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:56:57.753043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6e819-b8f8-4122-b751-a2bd15c6be14 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.761169 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c2090-2142-4590-ba1f-136409868c3c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.785095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:57.812322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a43b5-e578-4a0d-a82b-f1ec64532bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.812966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a43b5-e578-4a0d-a82b-f1ec64532bce service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.813245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a43b5-e578-4a0d-a82b-f1ec64532bce service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.813513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a43b5-e578-4a0d-a82b-f1ec64532bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.819743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6e819-b8f8-4122-b751-a2bd15c6be14 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.824966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97650ff0-aff7-481a-baa1-c4e75a2e037f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.825575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97650ff0-aff7-481a-baa1-c4e75a2e037f admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.825861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97650ff0-aff7-481a-baa1-c4e75a2e037f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.826132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97650ff0-aff7-481a-baa1-c4e75a2e037f admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.826795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97650ff0-aff7-481a-baa1-c4e75a2e037f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.828748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97650ff0-aff7-481a-baa1-c4e75a2e037f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.854316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 884/1768] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:56:57 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:56:57.855635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97650ff0-aff7-481a-baa1-c4e75a2e037f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.866715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97650ff0-aff7-481a-baa1-c4e75a2e037f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.872888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a9599-e30a-4d77-a00a-6bbec5be4a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.873505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a9599-e30a-4d77-a00a-6bbec5be4a06 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.873788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a9599-e30a-4d77-a00a-6bbec5be4a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.874062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a9599-e30a-4d77-a00a-6bbec5be4a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.874740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a9599-e30a-4d77-a00a-6bbec5be4a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.876309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a9599-e30a-4d77-a00a-6bbec5be4a06 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.883379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a43b5-e578-4a0d-a82b-f1ec64532bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.884871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a43b5-e578-4a0d-a82b-f1ec64532bce service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.887473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 885/1769] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:57.891540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a9599-e30a-4d77-a00a-6bbec5be4a06 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.901790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 885/1770] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:56:57.903717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a9599-e30a-4d77-a00a-6bbec5be4a06 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.908740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba243e-0b44-46e7-9460-7473e0a7d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.909387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba243e-0b44-46e7-9460-7473e0a7d266 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.909668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba243e-0b44-46e7-9460-7473e0a7d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.909943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba243e-0b44-46e7-9460-7473e0a7d266 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.910758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba243e-0b44-46e7-9460-7473e0a7d266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.913227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba243e-0b44-46e7-9460-7473e0a7d266 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:57.926223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 886/1771] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:56:57.930927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a43b5-e578-4a0d-a82b-f1ec64532bce service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:57.943993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a43b5-e578-4a0d-a82b-f1ec64532bce service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:57.953328 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c497171-5e30-49bb-b0bb-c09a30dd3bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:57.954220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c497171-5e30-49bb-b0bb-c09a30dd3bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:57.954651 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c497171-5e30-49bb-b0bb-c09a30dd3bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:57.955075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c497171-5e30-49bb-b0bb-c09a30dd3bc5 admin admin] PATH_INFO: `/v3/projects/ef86221762794317a02d4204bbfc5731/users/a404b60e687b4d4187884515572bee47/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:57.985623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c497171-5e30-49bb-b0bb-c09a30dd3bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef86221762794317a02d4204bbfc5731, user_id=a404b60e687b4d4187884515572bee47, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:57.988526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c497171-5e30-49bb-b0bb-c09a30dd3bc5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:58.018591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 886/1772] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:57 2022] PUT /identity/v3/projects/ef86221762794317a02d4204bbfc5731/users/a404b60e687b4d4187884515572bee47/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:58.020680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3f2f4-b9da-4b96-bb0e-d9c67fb8b8ba None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.021117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3f2f4-b9da-4b96-bb0e-d9c67fb8b8ba None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.021704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3f2f4-b9da-4b96-bb0e-d9c67fb8b8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.073228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2eb3f2f4-b9da-4b96-bb0e-d9c67fb8b8ba None None] MFA Rules not processed for user `a404b60e687b4d4187884515572bee47`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:58.073228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2eb3f2f4-b9da-4b96-bb0e-d9c67fb8b8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:58.174369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 887/1773] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:56:58.175702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c497171-5e30-49bb-b0bb-c09a30dd3bc5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:58.259241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:58.286621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c497171-5e30-49bb-b0bb-c09a30dd3bc5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:58.293709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98348d-9307-4086-b94e-b4f4e8aae88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:58.294611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98348d-9307-4086-b94e-b4f4e8aae88d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.295008 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98348d-9307-4086-b94e-b4f4e8aae88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.295174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98348d-9307-4086-b94e-b4f4e8aae88d admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.295817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98348d-9307-4086-b94e-b4f4e8aae88d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:58.297813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98348d-9307-4086-b94e-b4f4e8aae88d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:58.345799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 887/1774] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:56:58 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:56:58.347482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3f2f4-b9da-4b96-bb0e-d9c67fb8b8ba None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:58.359640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3f2f4-b9da-4b96-bb0e-d9c67fb8b8ba None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:58.414377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98348d-9307-4086-b94e-b4f4e8aae88d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:58.425208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b9272-e981-4aae-961b-55ffdacae749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:58.425827 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b9272-e981-4aae-961b-55ffdacae749 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.426108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b9272-e981-4aae-961b-55ffdacae749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.426380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b9272-e981-4aae-961b-55ffdacae749 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.427144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b9272-e981-4aae-961b-55ffdacae749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:58.429613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b9272-e981-4aae-961b-55ffdacae749 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:58.454263 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 888/1775] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:56:58 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:56:58.455394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b9272-e981-4aae-961b-55ffdacae749 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:58.466649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b9272-e981-4aae-961b-55ffdacae749 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:58.472292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b946a5-2338-4c66-98d6-34bdc33e27ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:58.473125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b946a5-2338-4c66-98d6-34bdc33e27ce admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.473408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b946a5-2338-4c66-98d6-34bdc33e27ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.473689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b946a5-2338-4c66-98d6-34bdc33e27ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.474975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b946a5-2338-4c66-98d6-34bdc33e27ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:58.476749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b946a5-2338-4c66-98d6-34bdc33e27ce admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:58.479693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98348d-9307-4086-b94e-b4f4e8aae88d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:58.484403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:58.485071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.485377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.485649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] PATH_INFO: `/v3/users/b9660451b42142aaad441e9966f1276c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.486788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9660451b42142aaad441e9966f1276c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:58.487052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 889/1776] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:56:58.488727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:58.491051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b946a5-2338-4c66-98d6-34bdc33e27ce admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:58.497043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:58.500871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b946a5-2338-4c66-98d6-34bdc33e27ce admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:58.507347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7db6f-66fc-4f4f-8c82-54db777e308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:58.508077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7db6f-66fc-4f4f-8c82-54db777e308f admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.508363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7db6f-66fc-4f4f-8c82-54db777e308f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.508633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7db6f-66fc-4f4f-8c82-54db777e308f admin admin] PATH_INFO: `/v3/projects/0af9910d7ea242c3b14b788a4b165ad5/users/41af0315fa9442d3a9a1002ece87b780/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.528219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7db6f-66fc-4f4f-8c82-54db777e308f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af9910d7ea242c3b14b788a4b165ad5, user_id=41af0315fa9442d3a9a1002ece87b780, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:58.531068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7db6f-66fc-4f4f-8c82-54db777e308f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:58.545390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9660451b42142aaad441e9966f1276c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:58.550896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 890/1777] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:58 2022] PUT /identity/v3/projects/0af9910d7ea242c3b14b788a4b165ad5/users/41af0315fa9442d3a9a1002ece87b780/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:56:58.553365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9660451b42142aaad441e9966f1276c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:58.554564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9660451b42142aaad441e9966f1276c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:58.555453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7db6f-66fc-4f4f-8c82-54db777e308f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:58.562007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9660451b42142aaad441e9966f1276c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:58.562656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a605f151-ce62-4f54-9876-8a59487dbdfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9660451b42142aaad441e9966f1276c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:58.583766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 888/1778] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:58 2022] DELETE /identity/v3/users/b9660451b42142aaad441e9966f1276c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:56:58.588625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d9a2c-4c90-4986-a49b-dfa003ccd633 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.588952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d9a2c-4c90-4986-a49b-dfa003ccd633 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.589229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d9a2c-4c90-4986-a49b-dfa003ccd633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.608161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-830d9a2c-4c90-4986-a49b-dfa003ccd633 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:58.609470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-830d9a2c-4c90-4986-a49b-dfa003ccd633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:58.623112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7db6f-66fc-4f4f-8c82-54db777e308f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:58.654196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:58.677485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 889/1779] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:58.684031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d9a2c-4c90-4986-a49b-dfa003ccd633 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:58.685843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-830d9a2c-4c90-4986-a49b-dfa003ccd633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:58.686892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1838a68-cbe6-4ede-99b7-6d723bb0bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:58.687518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1838a68-cbe6-4ede-99b7-6d723bb0bd4b admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.687795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1838a68-cbe6-4ede-99b7-6d723bb0bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.688079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1838a68-cbe6-4ede-99b7-6d723bb0bd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.688704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1838a68-cbe6-4ede-99b7-6d723bb0bd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:58.690335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1838a68-cbe6-4ede-99b7-6d723bb0bd4b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:58.702246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 891/1780] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:56:58.706350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1838a68-cbe6-4ede-99b7-6d723bb0bd4b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:58.708103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a1838a68-cbe6-4ede-99b7-6d723bb0bd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:58.721083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1838a68-cbe6-4ede-99b7-6d723bb0bd4b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:58.726787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe35cbc-ad4a-40cb-a3d9-99a4c74ea6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:58.727440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe35cbc-ad4a-40cb-a3d9-99a4c74ea6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.727718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe35cbc-ad4a-40cb-a3d9-99a4c74ea6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.727989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe35cbc-ad4a-40cb-a3d9-99a4c74ea6fb admin admin] PATH_INFO: `/v3/projects/0af9910d7ea242c3b14b788a4b165ad5/users/41af0315fa9442d3a9a1002ece87b780/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.730058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe35cbc-ad4a-40cb-a3d9-99a4c74ea6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af9910d7ea242c3b14b788a4b165ad5, user_id=41af0315fa9442d3a9a1002ece87b780, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:58.732868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe35cbc-ad4a-40cb-a3d9-99a4c74ea6fb admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:58.744507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 892/1781] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:56:58 2022] PUT /identity/v3/projects/0af9910d7ea242c3b14b788a4b165ad5/users/41af0315fa9442d3a9a1002ece87b780/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:56:58.754841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab627684-288d-49da-9bd2-035c6f910c04 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.755158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab627684-288d-49da-9bd2-035c6f910c04 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.755432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab627684-288d-49da-9bd2-035c6f910c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.764327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d9a2c-4c90-4986-a49b-dfa003ccd633 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:58.789564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ab627684-288d-49da-9bd2-035c6f910c04 None None] MFA Rules not processed for user `41af0315fa9442d3a9a1002ece87b780`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:56:58.790893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ab627684-288d-49da-9bd2-035c6f910c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:58.834396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e952ca-400d-4164-961f-e556ce6720d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:58.835793 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e952ca-400d-4164-961f-e556ce6720d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.836075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e952ca-400d-4164-961f-e556ce6720d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.836352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e952ca-400d-4164-961f-e556ce6720d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:58.870445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:58.887986 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 893/1782] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:56:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:56:58.897950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab627684-288d-49da-9bd2-035c6f910c04 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:58.899732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ab627684-288d-49da-9bd2-035c6f910c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:58.922368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e952ca-400d-4164-961f-e556ce6720d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:58.924397 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e952ca-400d-4164-961f-e556ce6720d3 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:58.938382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 890/1783] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:56:58.991132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab627684-288d-49da-9bd2-035c6f910c04 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:58.994993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19828417-ed83-4f71-a549-6587936b9178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:58.995642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19828417-ed83-4f71-a549-6587936b9178 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:58.995923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19828417-ed83-4f71-a549-6587936b9178 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:58.996196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19828417-ed83-4f71-a549-6587936b9178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:59.006544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19828417-ed83-4f71-a549-6587936b9178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:59.008206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19828417-ed83-4f71-a549-6587936b9178 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:59.018389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 894/1784] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:56:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:56:59.317837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e952ca-400d-4164-961f-e556ce6720d3 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:59.320097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-69e952ca-400d-4164-961f-e556ce6720d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:56:59.348401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e952ca-400d-4164-961f-e556ce6720d3 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:59.359786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:59.360716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:59.361179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:59.361603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] PATH_INFO: `/v3/projects/57194ea9a3544c0eb4eeac25f804a93d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:59.364077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=57194ea9a3544c0eb4eeac25f804a93d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:59.371058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:59.433846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57194ea9a3544c0eb4eeac25f804a93d', 'request_id': 'req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.435531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:56:59.442186 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57194ea9a3544c0eb4eeac25f804a93d', 'request_id': 'req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.452077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57194ea9a3544c0eb4eeac25f804a93d', 'request_id': 'req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.466140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] The token cache is being invalidate because project 57194ea9a3544c0eb4eeac25f804a93d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:56:59.466436 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.467108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.468861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 891/1785] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:56:59 2022] DELETE /identity/v3/projects/57194ea9a3544c0eb4eeac25f804a93d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:56:59.478283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19828417-ed83-4f71-a549-6587936b9178 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:56:59.495430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19828417-ed83-4f71-a549-6587936b9178 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:56:59.590334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:56:59.591100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:56:59.591464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:56:59.592078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] PATH_INFO: `/v3/users/4864f18b63ee4deaae44fbe24323f4e7` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:56:59.593368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4864f18b63ee4deaae44fbe24323f4e7)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:56:59.595636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:56:59.665916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4864f18b63ee4deaae44fbe24323f4e7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.669014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4864f18b63ee4deaae44fbe24323f4e7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.676503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4864f18b63ee4deaae44fbe24323f4e7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.678218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4864f18b63ee4deaae44fbe24323f4e7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.687382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3781dcba-a03d-44b9-a924-7ddae545fb77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4864f18b63ee4deaae44fbe24323f4e7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:56:59.705382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 895/1786] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:56:59 2022] DELETE /identity/v3/users/4864f18b63ee4deaae44fbe24323f4e7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:00.002219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:00.003669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:00.025793 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ccc90-9992-4ccc-abd5-ab4f7323a05e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:00.137907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:00.138481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.138904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.139188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] PATH_INFO: `/v3/projects/ac66ac23cde24c10b2b924afba0d58ff` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.140611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac66ac23cde24c10b2b924afba0d58ff)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:00.143113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:00.218390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac66ac23cde24c10b2b924afba0d58ff', 'request_id': 'req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.223892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac66ac23cde24c10b2b924afba0d58ff', 'request_id': 'req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.232301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac66ac23cde24c10b2b924afba0d58ff', 'request_id': 'req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.244629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] The token cache is being invalidate because project ac66ac23cde24c10b2b924afba0d58ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:57:00.244779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.245479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d82ed2df-ba46-48c5-8ad5-a8ca2bf29fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.246998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 892/1787] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:57:00 2022] DELETE /identity/v3/projects/ac66ac23cde24c10b2b924afba0d58ff => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:00.253363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d438-cf7c-4b37-9379-1709b4b7b306 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.253506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d438-cf7c-4b37-9379-1709b4b7b306 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.254010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d438-cf7c-4b37-9379-1709b4b7b306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.274800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9e76d438-cf7c-4b37-9379-1709b4b7b306 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:00.276026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9e76d438-cf7c-4b37-9379-1709b4b7b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:00.289732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:00.350163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 896/1788] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:57:00.358654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4165-ca01-48c1-a1c2-c2fa4a169cf0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.359139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4165-ca01-48c1-a1c2-c2fa4a169cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.359258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4165-ca01-48c1-a1c2-c2fa4a169cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.386982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0dec4165-ca01-48c1-a1c2-c2fa4a169cf0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:00.388416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0dec4165-ca01-48c1-a1c2-c2fa4a169cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:00.400834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 893/1789] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:00.402746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d438-cf7c-4b37-9379-1709b4b7b306 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:00.413199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d438-cf7c-4b37-9379-1709b4b7b306 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:00.419130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbda3b9-cd00-40c6-9f72-74f4e18fe111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:00.419750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbda3b9-cd00-40c6-9f72-74f4e18fe111 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.420186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbda3b9-cd00-40c6-9f72-74f4e18fe111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.420583 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbda3b9-cd00-40c6-9f72-74f4e18fe111 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.421270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbda3b9-cd00-40c6-9f72-74f4e18fe111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:57:00.421713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbda3b9-cd00-40c6-9f72-74f4e18fe111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:00.423939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbda3b9-cd00-40c6-9f72-74f4e18fe111 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:00.437740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 897/1790] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:57:00.442033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4165-ca01-48c1-a1c2-c2fa4a169cf0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:00.453115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4165-ca01-48c1-a1c2-c2fa4a169cf0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:00.459446 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd046552-9b1f-4945-ae4f-8e7fdce3350a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:00.460100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd046552-9b1f-4945-ae4f-8e7fdce3350a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.460369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd046552-9b1f-4945-ae4f-8e7fdce3350a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.460791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd046552-9b1f-4945-ae4f-8e7fdce3350a admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.461477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd046552-9b1f-4945-ae4f-8e7fdce3350a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:00.463399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd046552-9b1f-4945-ae4f-8e7fdce3350a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:00.503075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 894/1791] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:00 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:00.509993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbda3b9-cd00-40c6-9f72-74f4e18fe111 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:00.520485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbda3b9-cd00-40c6-9f72-74f4e18fe111 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:00.596512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a955562-e31d-480d-90f7-9b09752fbc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:00.597392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a955562-e31d-480d-90f7-9b09752fbc4d admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.597740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a955562-e31d-480d-90f7-9b09752fbc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.598118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a955562-e31d-480d-90f7-9b09752fbc4d admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.598970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a955562-e31d-480d-90f7-9b09752fbc4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:00.601661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a955562-e31d-480d-90f7-9b09752fbc4d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:00.634174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd046552-9b1f-4945-ae4f-8e7fdce3350a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:00.635016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fd046552-9b1f-4945-ae4f-8e7fdce3350a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:00.640204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 898/1792] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:00 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:00.646512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a955562-e31d-480d-90f7-9b09752fbc4d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:00.651403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd046552-9b1f-4945-ae4f-8e7fdce3350a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:00.656993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a955562-e31d-480d-90f7-9b09752fbc4d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:00.658254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:00.659234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.659644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.660056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] PATH_INFO: `/v3/users/a404b60e687b4d4187884515572bee47` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.661773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a404b60e687b4d4187884515572bee47)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:00.662092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13343308-95a6-4d80-aa8a-6739be945956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:00.663136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13343308-95a6-4d80-aa8a-6739be945956 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.663628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13343308-95a6-4d80-aa8a-6739be945956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.663972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13343308-95a6-4d80-aa8a-6739be945956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.664849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13343308-95a6-4d80-aa8a-6739be945956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:00.665059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:00.666706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13343308-95a6-4d80-aa8a-6739be945956 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:00.680835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 899/1793] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:57:00.687102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13343308-95a6-4d80-aa8a-6739be945956 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:00.717193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13343308-95a6-4d80-aa8a-6739be945956 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:00.722718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806de59-e617-41ca-a23b-97f8d3c0664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:00.723645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806de59-e617-41ca-a23b-97f8d3c0664d admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.724074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806de59-e617-41ca-a23b-97f8d3c0664d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.724531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806de59-e617-41ca-a23b-97f8d3c0664d admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.725522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806de59-e617-41ca-a23b-97f8d3c0664d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:00.726978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806de59-e617-41ca-a23b-97f8d3c0664d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:00.741709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 900/1794] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:00.745389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806de59-e617-41ca-a23b-97f8d3c0664d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:00.754063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:00.758427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806de59-e617-41ca-a23b-97f8d3c0664d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:00.763780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902badd-640d-4fa8-b999-366fc7edbe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:00.764464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902badd-640d-4fa8-b999-366fc7edbe65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.765135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902badd-640d-4fa8-b999-366fc7edbe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.765135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902badd-640d-4fa8-b999-366fc7edbe65 admin admin] PATH_INFO: `/v3/projects/47362556b66f405fbedb849e6d0f0b9f/users/c1c8a563a78546fe9714d69913418c6b/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.767710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a404b60e687b4d4187884515572bee47'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.782660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a404b60e687b4d4187884515572bee47'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.783363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a404b60e687b4d4187884515572bee47'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.790096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902badd-640d-4fa8-b999-366fc7edbe65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47362556b66f405fbedb849e6d0f0b9f, user_id=c1c8a563a78546fe9714d69913418c6b, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:00.794020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902badd-640d-4fa8-b999-366fc7edbe65 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:00.809681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a404b60e687b4d4187884515572bee47'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.811124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4de88f25-3549-4ed2-9809-bc7f82b3a6a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a404b60e687b4d4187884515572bee47'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:00.816584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 901/1795] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:00 2022] PUT /identity/v3/projects/47362556b66f405fbedb849e6d0f0b9f/users/c1c8a563a78546fe9714d69913418c6b/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:57:00.823855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6623f14-5ad6-4315-9ac9-263748957f85 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.824577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6623f14-5ad6-4315-9ac9-263748957f85 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.824884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6623f14-5ad6-4315-9ac9-263748957f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.872832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e6623f14-5ad6-4315-9ac9-263748957f85 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:00.872832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e6623f14-5ad6-4315-9ac9-263748957f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:00.885924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 895/1796] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:57:00 2022] DELETE /identity/v3/users/a404b60e687b4d4187884515572bee47 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:00.888711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaae3-5a8e-43cd-9cb3-99c8a8bbc03b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:00.890178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaae3-5a8e-43cd-9cb3-99c8a8bbc03b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:00.890466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaae3-5a8e-43cd-9cb3-99c8a8bbc03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:00.914142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-13faaae3-5a8e-43cd-9cb3-99c8a8bbc03b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:00.915451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-13faaae3-5a8e-43cd-9cb3-99c8a8bbc03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:00.949296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:00.967199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 902/1797] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:00.978452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6623f14-5ad6-4315-9ac9-263748957f85 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:00.979853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e6623f14-5ad6-4315-9ac9-263748957f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:00.998821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 896/1798] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:57:01.011009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faaae3-5a8e-43cd-9cb3-99c8a8bbc03b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:01.012854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-13faaae3-5a8e-43cd-9cb3-99c8a8bbc03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:01.049914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6623f14-5ad6-4315-9ac9-263748957f85 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:01.075864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faaae3-5a8e-43cd-9cb3-99c8a8bbc03b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:01.080417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed813cf-5ce8-46d8-9d60-dc8817abbb89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:01.081065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed813cf-5ce8-46d8-9d60-dc8817abbb89 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:01.081344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed813cf-5ce8-46d8-9d60-dc8817abbb89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:01.081616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed813cf-5ce8-46d8-9d60-dc8817abbb89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:01.091857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed813cf-5ce8-46d8-9d60-dc8817abbb89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:01.093389 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed813cf-5ce8-46d8-9d60-dc8817abbb89 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:01.106995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 897/1799] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:57:01.120240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a28433-30e0-41be-b365-5a2d1c5dfd4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:01.120899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a28433-30e0-41be-b365-5a2d1c5dfd4d service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:01.121193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a28433-30e0-41be-b365-5a2d1c5dfd4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:01.121716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a28433-30e0-41be-b365-5a2d1c5dfd4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:01.131849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a28433-30e0-41be-b365-5a2d1c5dfd4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:01.133372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a28433-30e0-41be-b365-5a2d1c5dfd4d service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:01.143921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 903/1800] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:57:01.560575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed813cf-5ce8-46d8-9d60-dc8817abbb89 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:01.562592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9ed813cf-5ce8-46d8-9d60-dc8817abbb89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:01.577174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed813cf-5ce8-46d8-9d60-dc8817abbb89 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:01.582154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:01.582904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:01.583309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:01.584257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] PATH_INFO: `/v3/projects/ef86221762794317a02d4204bbfc5731` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:01.585547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef86221762794317a02d4204bbfc5731)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:01.587570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:01.643836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef86221762794317a02d4204bbfc5731', 'request_id': 'req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.645807 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:01.658254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef86221762794317a02d4204bbfc5731', 'request_id': 'req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.669436 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef86221762794317a02d4204bbfc5731', 'request_id': 'req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.682104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] The token cache is being invalidate because project ef86221762794317a02d4204bbfc5731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:57:01.682401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.683241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.688623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 898/1801] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:57:01 2022] DELETE /identity/v3/projects/ef86221762794317a02d4204bbfc5731 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:01.689831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a28433-30e0-41be-b365-5a2d1c5dfd4d service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:01.703478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a28433-30e0-41be-b365-5a2d1c5dfd4d service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:01.774082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:01.775537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:01.775771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:01.777138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] PATH_INFO: `/v3/users/41af0315fa9442d3a9a1002ece87b780` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:01.778540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41af0315fa9442d3a9a1002ece87b780)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:01.780581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:01.837442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41af0315fa9442d3a9a1002ece87b780'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.839560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41af0315fa9442d3a9a1002ece87b780'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.843914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:01.852341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41af0315fa9442d3a9a1002ece87b780'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.853338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41af0315fa9442d3a9a1002ece87b780'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.866132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ef3c622-69a8-4e11-87e6-5876f8485e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41af0315fa9442d3a9a1002ece87b780'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:01.909278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 904/1802] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:57:01 2022] DELETE /identity/v3/users/41af0315fa9442d3a9a1002ece87b780 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:02.897809 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:02.900305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:02.917846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f05b2-1a43-4dbf-ba97-5214ea9032c4 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:02.918371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f05b2-1a43-4dbf-ba97-5214ea9032c4 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:02.918371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f05b2-1a43-4dbf-ba97-5214ea9032c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:02.931433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8f77f-dcd9-4d11-ba71-ef9d6ca34550 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:02.991679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-448f05b2-1a43-4dbf-ba97-5214ea9032c4 None None] MFA Rules not processed for user `c1c8a563a78546fe9714d69913418c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:02.993569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-448f05b2-1a43-4dbf-ba97-5214ea9032c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:03.066788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:03.067454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.067731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.068918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] PATH_INFO: `/v3/projects/0af9910d7ea242c3b14b788a4b165ad5` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.074161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af9910d7ea242c3b14b788a4b165ad5)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:03.077152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:03.141010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 905/1803] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:57:03.144591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af9910d7ea242c3b14b788a4b165ad5', 'request_id': 'req-0be981bc-62d8-4781-8516-afe46ffce58a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:03.149540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f05b2-1a43-4dbf-ba97-5214ea9032c4 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:03.150979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-448f05b2-1a43-4dbf-ba97-5214ea9032c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:03.152574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af9910d7ea242c3b14b788a4b165ad5', 'request_id': 'req-0be981bc-62d8-4781-8516-afe46ffce58a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:03.165199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af9910d7ea242c3b14b788a4b165ad5', 'request_id': 'req-0be981bc-62d8-4781-8516-afe46ffce58a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:03.179281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] The token cache is being invalidate because project 0af9910d7ea242c3b14b788a4b165ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:57:03.179503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:03.180459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0be981bc-62d8-4781-8516-afe46ffce58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:03.185292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 899/1804] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:57:02 2022] DELETE /identity/v3/projects/0af9910d7ea242c3b14b788a4b165ad5 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:03.189270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3944c-2a5b-497b-8ac6-8bb17766cd09 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.189555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3944c-2a5b-497b-8ac6-8bb17766cd09 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.189841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3944c-2a5b-497b-8ac6-8bb17766cd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.221267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-89c3944c-2a5b-497b-8ac6-8bb17766cd09 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:03.222664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-89c3944c-2a5b-497b-8ac6-8bb17766cd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:03.270309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f05b2-1a43-4dbf-ba97-5214ea9032c4 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:03.363366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 900/1805] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:57:03.366506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36d044-915b-4dc2-87ef-70c0d400ca45 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.366831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36d044-915b-4dc2-87ef-70c0d400ca45 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.367121 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36d044-915b-4dc2-87ef-70c0d400ca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.387119 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d387e-e091-4c48-b793-ead1eac432ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:03.387752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d387e-e091-4c48-b793-ead1eac432ea service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.388486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d387e-e091-4c48-b793-ead1eac432ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.388486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d387e-e091-4c48-b793-ead1eac432ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.390591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9c1d387e-e091-4c48-b793-ead1eac432ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:03.422666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bf36d044-915b-4dc2-87ef-70c0d400ca45 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:03.424056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bf36d044-915b-4dc2-87ef-70c0d400ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:03.436410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 901/1806] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:03.440308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36d044-915b-4dc2-87ef-70c0d400ca45 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:03.443301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:03.446670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:03.453378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36d044-915b-4dc2-87ef-70c0d400ca45 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:03.461359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29865fb-216d-44d4-bb3a-431ddd39d853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:03.461989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29865fb-216d-44d4-bb3a-431ddd39d853 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.463647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29865fb-216d-44d4-bb3a-431ddd39d853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.464728 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29865fb-216d-44d4-bb3a-431ddd39d853 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.465836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29865fb-216d-44d4-bb3a-431ddd39d853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:57:03.466072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29865fb-216d-44d4-bb3a-431ddd39d853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:03.468063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29865fb-216d-44d4-bb3a-431ddd39d853 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:03.482408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d387e-e091-4c48-b793-ead1eac432ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:03.484140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d387e-e091-4c48-b793-ead1eac432ea service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:03.484602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 902/1807] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:57:03.489080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29865fb-216d-44d4-bb3a-431ddd39d853 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:03.494736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 906/1808] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:57:03.501952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29865fb-216d-44d4-bb3a-431ddd39d853 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:03.508213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eeddc6-0c9a-4708-a17e-0a524546e184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:03.509102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeddc6-0c9a-4708-a17e-0a524546e184 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.509553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeddc6-0c9a-4708-a17e-0a524546e184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.510008 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeddc6-0c9a-4708-a17e-0a524546e184 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.511221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eeddc6-0c9a-4708-a17e-0a524546e184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:03.514979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eeddc6-0c9a-4708-a17e-0a524546e184 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:03.552562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 903/1809] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:03 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:03.556683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d387e-e091-4c48-b793-ead1eac432ea service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:03.570076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d387e-e091-4c48-b793-ead1eac432ea service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:03.673048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988976ef-a2f0-42e7-af57-93a201057b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:03.673766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988976ef-a2f0-42e7-af57-93a201057b82 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.674055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988976ef-a2f0-42e7-af57-93a201057b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.674332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988976ef-a2f0-42e7-af57-93a201057b82 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.675062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988976ef-a2f0-42e7-af57-93a201057b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:03.678067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988976ef-a2f0-42e7-af57-93a201057b82 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:03.731811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 907/1810] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:03 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:57:03.733294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eeddc6-0c9a-4708-a17e-0a524546e184 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:03.745971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eeddc6-0c9a-4708-a17e-0a524546e184 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:03.761022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de810-b4ef-4475-9d0d-1f46d3c005d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:03.761650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de810-b4ef-4475-9d0d-1f46d3c005d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.761931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de810-b4ef-4475-9d0d-1f46d3c005d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.762260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de810-b4ef-4475-9d0d-1f46d3c005d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.762906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3de810-b4ef-4475-9d0d-1f46d3c005d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:03.764774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3de810-b4ef-4475-9d0d-1f46d3c005d2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:03.779868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 904/1811] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:03.784111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988976ef-a2f0-42e7-af57-93a201057b82 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:03.800003 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988976ef-a2f0-42e7-af57-93a201057b82 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:03.812217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544654e-8244-4048-8a17-34e04b4c8331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:03.813289 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544654e-8244-4048-8a17-34e04b4c8331 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.813289 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544654e-8244-4048-8a17-34e04b4c8331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.813501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544654e-8244-4048-8a17-34e04b4c8331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.814493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544654e-8244-4048-8a17-34e04b4c8331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:03.816122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544654e-8244-4048-8a17-34e04b4c8331 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:03.834572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 908/1812] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:03.839042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de810-b4ef-4475-9d0d-1f46d3c005d2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:03.852757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de810-b4ef-4475-9d0d-1f46d3c005d2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:03.864044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e75d3-8262-4a69-8ece-69b049a34d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:03.865425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e75d3-8262-4a69-8ece-69b049a34d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.865849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e75d3-8262-4a69-8ece-69b049a34d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.866120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e75d3-8262-4a69-8ece-69b049a34d33 admin admin] PATH_INFO: `/v3/projects/f41c8da8bcc84477a3d934d66905d876/users/32b1a1ad906744f69e3b410c0045d524/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.898260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e75d3-8262-4a69-8ece-69b049a34d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41c8da8bcc84477a3d934d66905d876, user_id=32b1a1ad906744f69e3b410c0045d524, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:03.902039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e75d3-8262-4a69-8ece-69b049a34d33 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:03.927921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 905/1813] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:03 2022] PUT /identity/v3/projects/f41c8da8bcc84477a3d934d66905d876/users/32b1a1ad906744f69e3b410c0045d524/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:03.941822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48970ce-759c-4b71-bef5-e6d8ad70fb86 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:03.942278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48970ce-759c-4b71-bef5-e6d8ad70fb86 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:03.942370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48970ce-759c-4b71-bef5-e6d8ad70fb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:03.990509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d48970ce-759c-4b71-bef5-e6d8ad70fb86 None None] MFA Rules not processed for user `32b1a1ad906744f69e3b410c0045d524`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:03.992170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d48970ce-759c-4b71-bef5-e6d8ad70fb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:04.080839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 909/1814] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:04.102796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e75d3-8262-4a69-8ece-69b049a34d33 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:04.104998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f93e75d3-8262-4a69-8ece-69b049a34d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:04.192486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e75d3-8262-4a69-8ece-69b049a34d33 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:04.196875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cb3e5-5685-4731-824c-2827271ac1c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:04.197795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb3e5-5685-4731-824c-2827271ac1c7 service glance] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:04.198314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb3e5-5685-4731-824c-2827271ac1c7 service glance] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:04.200889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb3e5-5685-4731-824c-2827271ac1c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:04.221320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48970ce-759c-4b71-bef5-e6d8ad70fb86 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:04.223089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d48970ce-759c-4b71-bef5-e6d8ad70fb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:04.224758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cb3e5-5685-4731-824c-2827271ac1c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:04.226797 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cb3e5-5685-4731-824c-2827271ac1c7 service glance] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:04.241052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 906/1815] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:57:04.273443 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:04.304626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48970ce-759c-4b71-bef5-e6d8ad70fb86 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:04.309232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:04.309856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:04.310131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:04.310401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] PATH_INFO: `/v3/users/2e97da4529d140358e92175d966d838d` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:04.311556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e97da4529d140358e92175d966d838d)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:04.313514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:04.371213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e97da4529d140358e92175d966d838d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:04.371911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e97da4529d140358e92175d966d838d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:04.381467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e97da4529d140358e92175d966d838d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:04.382167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e97da4529d140358e92175d966d838d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:04.391890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e97da4529d140358e92175d966d838d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:04.414204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 910/1816] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:57:04 2022] DELETE /identity/v3/users/2e97da4529d140358e92175d966d838d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:57:04.418588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306b09e-8bd1-424c-b1db-72eb48a93a28 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:04.419235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306b09e-8bd1-424c-b1db-72eb48a93a28 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:04.419747 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306b09e-8bd1-424c-b1db-72eb48a93a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:04.451592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1306b09e-8bd1-424c-b1db-72eb48a93a28 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:04.453136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1306b09e-8bd1-424c-b1db-72eb48a93a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:04.537702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 907/1817] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:57:04.552733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:04.557373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:04.659416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293422a1-f712-45b1-bcf2-bc897ac3f012 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:04.663229 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99312ca-ac0f-4a13-a6f7-94192bc3c3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:04.663914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99312ca-ac0f-4a13-a6f7-94192bc3c3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:04.664194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99312ca-ac0f-4a13-a6f7-94192bc3c3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:04.664472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99312ca-ac0f-4a13-a6f7-94192bc3c3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:04.673802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99312ca-ac0f-4a13-a6f7-94192bc3c3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:04.675224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99312ca-ac0f-4a13-a6f7-94192bc3c3a2 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:04.685120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 911/1818] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:57:05.100682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc4b37-d9b2-40d2-b4bd-fbacee0ccac8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:05.101263 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc4b37-d9b2-40d2-b4bd-fbacee0ccac8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:05.101380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc4b37-d9b2-40d2-b4bd-fbacee0ccac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:05.122538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-abcc4b37-d9b2-40d2-b4bd-fbacee0ccac8 None None] MFA Rules not processed for user `c1c8a563a78546fe9714d69913418c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:05.123675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-abcc4b37-d9b2-40d2-b4bd-fbacee0ccac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:05.133910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:05.198595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 908/1819] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:05.244942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99312ca-ac0f-4a13-a6f7-94192bc3c3a2 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:05.248153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c99312ca-ac0f-4a13-a6f7-94192bc3c3a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:05.273734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99312ca-ac0f-4a13-a6f7-94192bc3c3a2 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:05.295581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:05.296633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:05.296956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:05.297230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] PATH_INFO: `/v3/projects/1a62540e4d73465c8a17611589370ddd` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:05.298414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a62540e4d73465c8a17611589370ddd)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:05.300425 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:05.385353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a62540e4d73465c8a17611589370ddd', 'request_id': 'req-2e70f595-c159-41f2-9e35-5929876d9926'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.387562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:05.393209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a62540e4d73465c8a17611589370ddd', 'request_id': 'req-2e70f595-c159-41f2-9e35-5929876d9926'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.404247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a62540e4d73465c8a17611589370ddd', 'request_id': 'req-2e70f595-c159-41f2-9e35-5929876d9926'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.417127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] The token cache is being invalidate because project 1a62540e4d73465c8a17611589370ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:57:05.417558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.418097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.419893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 912/1820] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:57:05 2022] DELETE /identity/v3/projects/1a62540e4d73465c8a17611589370ddd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:05.423751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc4b37-d9b2-40d2-b4bd-fbacee0ccac8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:05.437150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc4b37-d9b2-40d2-b4bd-fbacee0ccac8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:05.528273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:05.528870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:05.529153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:05.529470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] PATH_INFO: `/v3/users/ac7a9de9241f446d97be98d331992584` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:05.531175 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac7a9de9241f446d97be98d331992584)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:05.533512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:05.600727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac7a9de9241f446d97be98d331992584'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.610548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac7a9de9241f446d97be98d331992584'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.611287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac7a9de9241f446d97be98d331992584'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.619960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac7a9de9241f446d97be98d331992584'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.620691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-25e07987-fccf-48d3-be7c-ac7934d14a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac7a9de9241f446d97be98d331992584'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:05.653925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 909/1821] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:57:05 2022] DELETE /identity/v3/users/ac7a9de9241f446d97be98d331992584 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:57:06.395033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:06.397515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:06.415588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70f595-c159-41f2-9e35-5929876d9926 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:06.582606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:06.584782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:06.585420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:06.585914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] PATH_INFO: `/v3/projects/27d3efabbdd54c8ebf551475995805a4` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:06.587566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d3efabbdd54c8ebf551475995805a4)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:06.591142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:06.676526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d3efabbdd54c8ebf551475995805a4', 'request_id': 'req-21a6530c-86ff-42d6-94df-15454ce391b5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:06.686091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d3efabbdd54c8ebf551475995805a4', 'request_id': 'req-21a6530c-86ff-42d6-94df-15454ce391b5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:06.700131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d3efabbdd54c8ebf551475995805a4', 'request_id': 'req-21a6530c-86ff-42d6-94df-15454ce391b5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:06.725849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] The token cache is being invalidate because project 27d3efabbdd54c8ebf551475995805a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:57:06.726606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:06.731053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-21a6530c-86ff-42d6-94df-15454ce391b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:06.737278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 913/1822] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:57:06 2022] DELETE /identity/v3/projects/27d3efabbdd54c8ebf551475995805a4 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:06.738354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eaf43-df90-49ee-aaf1-e22f0b423609 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:06.738849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eaf43-df90-49ee-aaf1-e22f0b423609 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:06.739190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eaf43-df90-49ee-aaf1-e22f0b423609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:06.799283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-835eaf43-df90-49ee-aaf1-e22f0b423609 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:06.800598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-835eaf43-df90-49ee-aaf1-e22f0b423609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:06.859351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:06.890780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 910/1823] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:57:06.898967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893bf35-a6b0-4e10-82df-0ab5cf61333a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:06.899476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893bf35-a6b0-4e10-82df-0ab5cf61333a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:06.899809 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893bf35-a6b0-4e10-82df-0ab5cf61333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:06.930013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-1893bf35-a6b0-4e10-82df-0ab5cf61333a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:06.931411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-1893bf35-a6b0-4e10-82df-0ab5cf61333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:06.940201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 914/1824] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:06.944531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835eaf43-df90-49ee-aaf1-e22f0b423609 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:06.954611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835eaf43-df90-49ee-aaf1-e22f0b423609 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:06.959854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c792f9-c63f-47e1-a09e-992cad9f010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:06.960460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c792f9-c63f-47e1-a09e-992cad9f010f admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:06.960743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c792f9-c63f-47e1-a09e-992cad9f010f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:06.961051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c792f9-c63f-47e1-a09e-992cad9f010f admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:06.961716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c792f9-c63f-47e1-a09e-992cad9f010f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:57:06.962020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c792f9-c63f-47e1-a09e-992cad9f010f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:06.963626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c792f9-c63f-47e1-a09e-992cad9f010f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:06.975930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 911/1825] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:57:06.980367 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893bf35-a6b0-4e10-82df-0ab5cf61333a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:06.990518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893bf35-a6b0-4e10-82df-0ab5cf61333a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:06.995210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032c2ae-29a4-427f-b2f0-0cdfb95940ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:06.995820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032c2ae-29a4-427f-b2f0-0cdfb95940ad admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:06.996100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032c2ae-29a4-427f-b2f0-0cdfb95940ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:06.996371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032c2ae-29a4-427f-b2f0-0cdfb95940ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:06.997030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032c2ae-29a4-427f-b2f0-0cdfb95940ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:06.999029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032c2ae-29a4-427f-b2f0-0cdfb95940ad admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.030058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 915/1826] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:06 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:07.031324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c792f9-c63f-47e1-a09e-992cad9f010f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.043543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c792f9-c63f-47e1-a09e-992cad9f010f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.113390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ec311-f1f2-443c-95d5-f26663dd1f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.114015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ec311-f1f2-443c-95d5-f26663dd1f35 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.114296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ec311-f1f2-443c-95d5-f26663dd1f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.115155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ec311-f1f2-443c-95d5-f26663dd1f35 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.115777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ec311-f1f2-443c-95d5-f26663dd1f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.117760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ec311-f1f2-443c-95d5-f26663dd1f35 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.154657 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 912/1827] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:07 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:07.156133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032c2ae-29a4-427f-b2f0-0cdfb95940ad admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.174125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032c2ae-29a4-427f-b2f0-0cdfb95940ad admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.180015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174d4a3-6dd3-4f5a-b500-050890eb1f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.180637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174d4a3-6dd3-4f5a-b500-050890eb1f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.180996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174d4a3-6dd3-4f5a-b500-050890eb1f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.181276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174d4a3-6dd3-4f5a-b500-050890eb1f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.181922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174d4a3-6dd3-4f5a-b500-050890eb1f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.183532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174d4a3-6dd3-4f5a-b500-050890eb1f19 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.197322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 916/1828] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:57:07.198790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ec311-f1f2-443c-95d5-f26663dd1f35 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.211081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ec311-f1f2-443c-95d5-f26663dd1f35 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.216012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41faecb-52c1-44b1-93ae-63864a82584d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.216631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41faecb-52c1-44b1-93ae-63864a82584d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.217020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41faecb-52c1-44b1-93ae-63864a82584d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.217298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41faecb-52c1-44b1-93ae-63864a82584d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.217944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41faecb-52c1-44b1-93ae-63864a82584d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.219542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41faecb-52c1-44b1-93ae-63864a82584d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.237042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 913/1829] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:07.237991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174d4a3-6dd3-4f5a-b500-050890eb1f19 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.251189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174d4a3-6dd3-4f5a-b500-050890eb1f19 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.256234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caa042-e355-4f96-9e82-4c6d3a46f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.260336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caa042-e355-4f96-9e82-4c6d3a46f4e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.260704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caa042-e355-4f96-9e82-4c6d3a46f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.261123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caa042-e355-4f96-9e82-4c6d3a46f4e7 admin admin] PATH_INFO: `/v3/projects/61b09f1eda6b4e698fb359d4951674d9/users/8749cb6226ad4604bc59b0eac41c8098/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.280363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caa042-e355-4f96-9e82-4c6d3a46f4e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b09f1eda6b4e698fb359d4951674d9, user_id=8749cb6226ad4604bc59b0eac41c8098, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.283489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caa042-e355-4f96-9e82-4c6d3a46f4e7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.299859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 917/1830] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:07 2022] PUT /identity/v3/projects/61b09f1eda6b4e698fb359d4951674d9/users/8749cb6226ad4604bc59b0eac41c8098/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:07.311378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52685fe1-3eff-4f08-a178-2354198ea7dc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.311669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52685fe1-3eff-4f08-a178-2354198ea7dc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.311941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52685fe1-3eff-4f08-a178-2354198ea7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.342472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-52685fe1-3eff-4f08-a178-2354198ea7dc None None] MFA Rules not processed for user `8749cb6226ad4604bc59b0eac41c8098`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:07.343794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-52685fe1-3eff-4f08-a178-2354198ea7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:07.441005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 914/1831] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:07.445915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caa042-e355-4f96-9e82-4c6d3a46f4e7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.458129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:07.527963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caa042-e355-4f96-9e82-4c6d3a46f4e7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.532574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64bd66-4a84-4579-846e-f1f9d21a0946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.533320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64bd66-4a84-4579-846e-f1f9d21a0946 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.533601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64bd66-4a84-4579-846e-f1f9d21a0946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.533874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64bd66-4a84-4579-846e-f1f9d21a0946 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.534531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64bd66-4a84-4579-846e-f1f9d21a0946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.537098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64bd66-4a84-4579-846e-f1f9d21a0946 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.573653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 918/1832] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:07 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:57:07.577930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52685fe1-3eff-4f08-a178-2354198ea7dc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.589700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52685fe1-3eff-4f08-a178-2354198ea7dc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.623165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:07.651859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219fc4-eb62-4676-886f-2a68748f4d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.652512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219fc4-eb62-4676-886f-2a68748f4d6c admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.652771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219fc4-eb62-4676-886f-2a68748f4d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.653067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219fc4-eb62-4676-886f-2a68748f4d6c admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.653693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0219fc4-eb62-4676-886f-2a68748f4d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.655614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0219fc4-eb62-4676-886f-2a68748f4d6c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.676625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 915/1833] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:07 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:07.680677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64bd66-4a84-4579-846e-f1f9d21a0946 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.691085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64bd66-4a84-4579-846e-f1f9d21a0946 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.696356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3232c7-7a39-4703-8536-aa51146384b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.697087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3232c7-7a39-4703-8536-aa51146384b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.697369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3232c7-7a39-4703-8536-aa51146384b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.697643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3232c7-7a39-4703-8536-aa51146384b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.698341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3232c7-7a39-4703-8536-aa51146384b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.700257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3232c7-7a39-4703-8536-aa51146384b6 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.712140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 919/1834] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:07.716325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219fc4-eb62-4676-886f-2a68748f4d6c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.727195 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219fc4-eb62-4676-886f-2a68748f4d6c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.731845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06913a-020a-4acf-a55a-6e915237c2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.732531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06913a-020a-4acf-a55a-6e915237c2a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.732835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06913a-020a-4acf-a55a-6e915237c2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.733112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06913a-020a-4acf-a55a-6e915237c2a3 admin admin] PATH_INFO: `/v3/projects/0ce7fdd6f42d40248d4052167b0ee39f/users/33ddbc8eb7ac4e6ca216a310ff7e4d93/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.748598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06913a-020a-4acf-a55a-6e915237c2a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce7fdd6f42d40248d4052167b0ee39f, user_id=33ddbc8eb7ac4e6ca216a310ff7e4d93, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.751329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06913a-020a-4acf-a55a-6e915237c2a3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.765238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 916/1835] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:07 2022] PUT /identity/v3/projects/0ce7fdd6f42d40248d4052167b0ee39f/users/33ddbc8eb7ac4e6ca216a310ff7e4d93/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:07.772929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3232c7-7a39-4703-8536-aa51146384b6 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.792088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3232c7-7a39-4703-8536-aa51146384b6 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.856872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41a3a9-ba1b-4295-9e76-520ef477a3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.857423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41a3a9-ba1b-4295-9e76-520ef477a3ed admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.857708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41a3a9-ba1b-4295-9e76-520ef477a3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.857983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41a3a9-ba1b-4295-9e76-520ef477a3ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.858636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41a3a9-ba1b-4295-9e76-520ef477a3ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.860264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41a3a9-ba1b-4295-9e76-520ef477a3ed admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.870933 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 920/1836] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:07.875562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06913a-020a-4acf-a55a-6e915237c2a3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:07.887506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06913a-020a-4acf-a55a-6e915237c2a3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:07.892833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42021a09-b6b0-4342-881a-0802e7836fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:07.893580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42021a09-b6b0-4342-881a-0802e7836fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.893933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42021a09-b6b0-4342-881a-0802e7836fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.894269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42021a09-b6b0-4342-881a-0802e7836fe0 admin admin] PATH_INFO: `/v3/projects/0ce7fdd6f42d40248d4052167b0ee39f/users/33ddbc8eb7ac4e6ca216a310ff7e4d93/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.897029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42021a09-b6b0-4342-881a-0802e7836fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce7fdd6f42d40248d4052167b0ee39f, user_id=33ddbc8eb7ac4e6ca216a310ff7e4d93, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:07.908072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42021a09-b6b0-4342-881a-0802e7836fe0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:07.924384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 917/1837] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:07 2022] PUT /identity/v3/projects/0ce7fdd6f42d40248d4052167b0ee39f/users/33ddbc8eb7ac4e6ca216a310ff7e4d93/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:07.937135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf1f6b-d4a4-47f2-acf5-ebc5eddf271d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:07.938007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf1f6b-d4a4-47f2-acf5-ebc5eddf271d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:07.941376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf1f6b-d4a4-47f2-acf5-ebc5eddf271d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:07.980227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-8fbf1f6b-d4a4-47f2-acf5-ebc5eddf271d None None] MFA Rules not processed for user `33ddbc8eb7ac4e6ca216a310ff7e4d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:07.981480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8fbf1f6b-d4a4-47f2-acf5-ebc5eddf271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:08.078792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 921/1838] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:08.089311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42021a09-b6b0-4342-881a-0802e7836fe0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:08.091031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-42021a09-b6b0-4342-881a-0802e7836fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:08.149838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42021a09-b6b0-4342-881a-0802e7836fe0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:08.153524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba929ae4-7fef-4c61-a559-9ab8a1fa2c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:08.154293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba929ae4-7fef-4c61-a559-9ab8a1fa2c96 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:08.154593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba929ae4-7fef-4c61-a559-9ab8a1fa2c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:08.154896 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba929ae4-7fef-4c61-a559-9ab8a1fa2c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:08.215961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba929ae4-7fef-4c61-a559-9ab8a1fa2c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:08.217481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba929ae4-7fef-4c61-a559-9ab8a1fa2c96 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:08.226788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 918/1839] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:57:17.137425 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf1f6b-d4a4-47f2-acf5-ebc5eddf271d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:17.138867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8fbf1f6b-d4a4-47f2-acf5-ebc5eddf271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:17.146157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:17.152002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf1f6b-d4a4-47f2-acf5-ebc5eddf271d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:17.155922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0258d-cf7b-4dda-a615-2eedddc00626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:17.156525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0258d-cf7b-4dda-a615-2eedddc00626 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:17.156827 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0258d-cf7b-4dda-a615-2eedddc00626 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:17.157110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0258d-cf7b-4dda-a615-2eedddc00626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:17.167611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0258d-cf7b-4dda-a615-2eedddc00626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:17.169069 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0258d-cf7b-4dda-a615-2eedddc00626 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:17.182117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 922/1840] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:57:18.222300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:27.174520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:50.950666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba929ae4-7fef-4c61-a559-9ab8a1fa2c96 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:50.952180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ba929ae4-7fef-4c61-a559-9ab8a1fa2c96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:51.036847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba929ae4-7fef-4c61-a559-9ab8a1fa2c96 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:51.042947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:51.043644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:51.043923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:51.044193 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] PATH_INFO: `/v3/users/32b1a1ad906744f69e3b410c0045d524` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:51.045436 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b1a1ad906744f69e3b410c0045d524)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:51.047382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:51.118282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b1a1ad906744f69e3b410c0045d524'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.127653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b1a1ad906744f69e3b410c0045d524'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.128650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b1a1ad906744f69e3b410c0045d524'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.139331 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b1a1ad906744f69e3b410c0045d524'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.140142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b1a1ad906744f69e3b410c0045d524'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.167739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 919/1841] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:57:50 2022] DELETE /identity/v3/users/32b1a1ad906744f69e3b410c0045d524 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:51.174597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14f580-6d1e-4579-bfd0-2354657b0088 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:51.175185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14f580-6d1e-4579-bfd0-2354657b0088 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:51.175399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14f580-6d1e-4579-bfd0-2354657b0088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:51.195158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-8a14f580-6d1e-4579-bfd0-2354657b0088 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:51.196481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8a14f580-6d1e-4579-bfd0-2354657b0088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:51.259018 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 923/1842] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:51.268982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:51.270522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:51.334640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e6da1-e59d-4ff0-b666-9773454d2757 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:51.338296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bea3c-eb3a-4c04-b18b-804f67b53f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:51.338915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bea3c-eb3a-4c04-b18b-804f67b53f6f service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:51.339198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bea3c-eb3a-4c04-b18b-804f67b53f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:51.339478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bea3c-eb3a-4c04-b18b-804f67b53f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:51.348829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bea3c-eb3a-4c04-b18b-804f67b53f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:51.350595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bea3c-eb3a-4c04-b18b-804f67b53f6f service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:51.360866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 920/1843] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:57:51.712408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14f580-6d1e-4579-bfd0-2354657b0088 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:51.713980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8a14f580-6d1e-4579-bfd0-2354657b0088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:51.726530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14f580-6d1e-4579-bfd0-2354657b0088 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:51.731653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:51.732292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:51.732569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:51.732867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] PATH_INFO: `/v3/projects/f41c8da8bcc84477a3d934d66905d876` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:51.733986 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41c8da8bcc84477a3d934d66905d876)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:51.735908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:51.784088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41c8da8bcc84477a3d934d66905d876', 'request_id': 'req-10dc0c73-960f-4179-8dcb-e2285613e3ff'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.790190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41c8da8bcc84477a3d934d66905d876', 'request_id': 'req-10dc0c73-960f-4179-8dcb-e2285613e3ff'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.797520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41c8da8bcc84477a3d934d66905d876', 'request_id': 'req-10dc0c73-960f-4179-8dcb-e2285613e3ff'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.809987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] The token cache is being invalidate because project f41c8da8bcc84477a3d934d66905d876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:57:51.810276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.810908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-10dc0c73-960f-4179-8dcb-e2285613e3ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:51.812396 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 924/1844] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:57:51 2022] DELETE /identity/v3/projects/f41c8da8bcc84477a3d934d66905d876 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:51.817384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562eac1-0f09-4f0d-855f-8df8a299da73 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:51.817864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562eac1-0f09-4f0d-855f-8df8a299da73 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:51.818270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562eac1-0f09-4f0d-855f-8df8a299da73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:51.838167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4562eac1-0f09-4f0d-855f-8df8a299da73 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:51.839289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4562eac1-0f09-4f0d-855f-8df8a299da73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:51.856744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:51.900357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 921/1845] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:51.908108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741458-9261-4e7a-ae9a-478099658d68 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:51.908392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741458-9261-4e7a-ae9a-478099658d68 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:51.908664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741458-9261-4e7a-ae9a-478099658d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:51.929246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-21741458-9261-4e7a-ae9a-478099658d68 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:51.930431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-21741458-9261-4e7a-ae9a-478099658d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:51.938083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 925/1846] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:51.942200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562eac1-0f09-4f0d-855f-8df8a299da73 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:51.952861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562eac1-0f09-4f0d-855f-8df8a299da73 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:51.957432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21150c8-fc55-467f-86bc-b5b5bf764dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:51.958692 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21150c8-fc55-467f-86bc-b5b5bf764dca admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:51.959204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21150c8-fc55-467f-86bc-b5b5bf764dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:51.959501 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21150c8-fc55-467f-86bc-b5b5bf764dca admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:51.960153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21150c8-fc55-467f-86bc-b5b5bf764dca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:57:51.960454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21150c8-fc55-467f-86bc-b5b5bf764dca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:51.962091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21150c8-fc55-467f-86bc-b5b5bf764dca admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:51.973785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 922/1847] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:57:51.978086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21741458-9261-4e7a-ae9a-478099658d68 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:51.987412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21741458-9261-4e7a-ae9a-478099658d68 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:51.993074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5843915-7be4-4582-a847-5f1fc1bf0408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:51.993688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5843915-7be4-4582-a847-5f1fc1bf0408 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:51.993962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5843915-7be4-4582-a847-5f1fc1bf0408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:51.994273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5843915-7be4-4582-a847-5f1fc1bf0408 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:51.994914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5843915-7be4-4582-a847-5f1fc1bf0408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:51.996835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5843915-7be4-4582-a847-5f1fc1bf0408 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:52.028250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 926/1848] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:51 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:52.031960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21150c8-fc55-467f-86bc-b5b5bf764dca admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:52.042407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21150c8-fc55-467f-86bc-b5b5bf764dca admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:52.105527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52a68-8f4f-45a0-bf8f-11b2bbb0d5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:52.106127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52a68-8f4f-45a0-bf8f-11b2bbb0d5eb admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:52.106407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52a68-8f4f-45a0-bf8f-11b2bbb0d5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:52.106678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52a68-8f4f-45a0-bf8f-11b2bbb0d5eb admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:52.107317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a52a68-8f4f-45a0-bf8f-11b2bbb0d5eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:52.109277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a52a68-8f4f-45a0-bf8f-11b2bbb0d5eb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:52.132866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 923/1849] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:52 2022] POST /identity/v3/users => generated 281 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:52.136748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5843915-7be4-4582-a847-5f1fc1bf0408 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:52.148462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5843915-7be4-4582-a847-5f1fc1bf0408 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:52.154096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501be27-3539-444b-a19e-d9e6cdfdd7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:52.154812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501be27-3539-444b-a19e-d9e6cdfdd7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:52.155138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501be27-3539-444b-a19e-d9e6cdfdd7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:52.155507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501be27-3539-444b-a19e-d9e6cdfdd7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:52.156065 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501be27-3539-444b-a19e-d9e6cdfdd7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:52.157737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501be27-3539-444b-a19e-d9e6cdfdd7b3 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:52.173192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52a68-8f4f-45a0-bf8f-11b2bbb0d5eb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:52.179574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 927/1850] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:57:52.186069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52a68-8f4f-45a0-bf8f-11b2bbb0d5eb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:52.194060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc441b-c87c-43cf-98e5-45c7546988e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:52.194325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc441b-c87c-43cf-98e5-45c7546988e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:52.194607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc441b-c87c-43cf-98e5-45c7546988e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:52.194882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc441b-c87c-43cf-98e5-45c7546988e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:52.195543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc441b-c87c-43cf-98e5-45c7546988e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:52.197347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc441b-c87c-43cf-98e5-45c7546988e8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:52.213605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 924/1851] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:52.215020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501be27-3539-444b-a19e-d9e6cdfdd7b3 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:52.226783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501be27-3539-444b-a19e-d9e6cdfdd7b3 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:52.232706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bcb1a-3ee0-47c6-9bc8-e417578f5230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:52.233601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bcb1a-3ee0-47c6-9bc8-e417578f5230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:52.234182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bcb1a-3ee0-47c6-9bc8-e417578f5230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:52.234689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bcb1a-3ee0-47c6-9bc8-e417578f5230 admin admin] PATH_INFO: `/v3/projects/d52e183d78a1435599961bf87bac8a54/users/add2627f77c447fcafe54dc4892ff3a9/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:52.267865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bcb1a-3ee0-47c6-9bc8-e417578f5230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d52e183d78a1435599961bf87bac8a54, user_id=add2627f77c447fcafe54dc4892ff3a9, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:52.278163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bcb1a-3ee0-47c6-9bc8-e417578f5230 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:52.314889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce3cc7-1488-472e-b7af-d524faf33105 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:52.315330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce3cc7-1488-472e-b7af-d524faf33105 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:52.315707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce3cc7-1488-472e-b7af-d524faf33105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:52.319282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 928/1852] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:52 2022] PUT /identity/v3/projects/d52e183d78a1435599961bf87bac8a54/users/add2627f77c447fcafe54dc4892ff3a9/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:57:52.367139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1cce3cc7-1488-472e-b7af-d524faf33105 None None] MFA Rules not processed for user `add2627f77c447fcafe54dc4892ff3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:52.368943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1cce3cc7-1488-472e-b7af-d524faf33105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:52.494904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 925/1853] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:52.531167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bcb1a-3ee0-47c6-9bc8-e417578f5230 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:52.533499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fd2bcb1a-3ee0-47c6-9bc8-e417578f5230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:52.575284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:52.624346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bcb1a-3ee0-47c6-9bc8-e417578f5230 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:52.629039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ea17d-f74f-46e7-ab3b-634a870572dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:52.630942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ea17d-f74f-46e7-ab3b-634a870572dd service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:52.631392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ea17d-f74f-46e7-ab3b-634a870572dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:52.631663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ea17d-f74f-46e7-ab3b-634a870572dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:52.641834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ea17d-f74f-46e7-ab3b-634a870572dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:52.643342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ea17d-f74f-46e7-ab3b-634a870572dd service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:52.671867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce3cc7-1488-472e-b7af-d524faf33105 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:52.684189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 929/1854] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:57:52.703217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:52.759524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce3cc7-1488-472e-b7af-d524faf33105 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:52.765912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:52.766648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:52.766975 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:52.767193 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] PATH_INFO: `/v3/users/add2627f77c447fcafe54dc4892ff3a9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:52.768369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] RBAC: Authorizing `identity:delete_user(user_id=add2627f77c447fcafe54dc4892ff3a9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:52.771235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:52.857009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add2627f77c447fcafe54dc4892ff3a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:52.867477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add2627f77c447fcafe54dc4892ff3a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:52.868323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add2627f77c447fcafe54dc4892ff3a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:52.876302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add2627f77c447fcafe54dc4892ff3a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:52.877041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add2627f77c447fcafe54dc4892ff3a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:52.905050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 926/1855] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:57:52 2022] DELETE /identity/v3/users/add2627f77c447fcafe54dc4892ff3a9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:52.905186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b9898-2806-4c8d-b8ec-c87ae992a944 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:52.905186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b9898-2806-4c8d-b8ec-c87ae992a944 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:52.905186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b9898-2806-4c8d-b8ec-c87ae992a944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:52.930720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5e8b9898-2806-4c8d-b8ec-c87ae992a944 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:52.931945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5e8b9898-2806-4c8d-b8ec-c87ae992a944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:53.003449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 930/1856] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:53.017729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:53.019679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:53.080375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea850b-2792-461d-9922-38c88a50360f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:53.084322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd1142-8174-42d0-bb05-2b57eea2f791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:53.084974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd1142-8174-42d0-bb05-2b57eea2f791 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:53.085254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd1142-8174-42d0-bb05-2b57eea2f791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:53.087040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd1142-8174-42d0-bb05-2b57eea2f791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:53.095659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd1142-8174-42d0-bb05-2b57eea2f791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:53.097161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd1142-8174-42d0-bb05-2b57eea2f791 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:53.108956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 927/1857] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:57:53.425704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b9898-2806-4c8d-b8ec-c87ae992a944 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:53.427781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5e8b9898-2806-4c8d-b8ec-c87ae992a944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:53.440378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b9898-2806-4c8d-b8ec-c87ae992a944 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:53.446973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:53.447619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:53.448523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:53.448991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] PATH_INFO: `/v3/projects/d52e183d78a1435599961bf87bac8a54` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:53.450155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d52e183d78a1435599961bf87bac8a54)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:53.452103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:53.517557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd52e183d78a1435599961bf87bac8a54', 'request_id': 'req-3754053b-8970-499e-818f-3bee93b6a372'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:53.524313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd52e183d78a1435599961bf87bac8a54', 'request_id': 'req-3754053b-8970-499e-818f-3bee93b6a372'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:53.534442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd52e183d78a1435599961bf87bac8a54', 'request_id': 'req-3754053b-8970-499e-818f-3bee93b6a372'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:53.553223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] The token cache is being invalidate because project d52e183d78a1435599961bf87bac8a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:57:53.553663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:53.554539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3754053b-8970-499e-818f-3bee93b6a372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:53.556886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 931/1858] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:57:53 2022] DELETE /identity/v3/projects/d52e183d78a1435599961bf87bac8a54 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:53.562035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7653ac-2890-47d4-b5a6-db41f2bd3916 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:53.562622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7653ac-2890-47d4-b5a6-db41f2bd3916 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:53.562904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7653ac-2890-47d4-b5a6-db41f2bd3916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:53.584479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1c7653ac-2890-47d4-b5a6-db41f2bd3916 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:53.585801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1c7653ac-2890-47d4-b5a6-db41f2bd3916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:53.622271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:53.650292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 928/1859] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:53.658814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f6fd3-f515-4fdc-9954-c3265af8769c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:53.659185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f6fd3-f515-4fdc-9954-c3265af8769c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:53.659673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f6fd3-f515-4fdc-9954-c3265af8769c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:53.683971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-204f6fd3-f515-4fdc-9954-c3265af8769c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:53.685504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-204f6fd3-f515-4fdc-9954-c3265af8769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:53.695121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 932/1860] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:53.703222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7653ac-2890-47d4-b5a6-db41f2bd3916 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:53.716377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7653ac-2890-47d4-b5a6-db41f2bd3916 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:53.721741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee55951-7e2e-4ddd-93e1-edaba145f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:53.722364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55951-7e2e-4ddd-93e1-edaba145f932 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:53.722795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55951-7e2e-4ddd-93e1-edaba145f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:53.723079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55951-7e2e-4ddd-93e1-edaba145f932 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:53.723779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee55951-7e2e-4ddd-93e1-edaba145f932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:57:53.724184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee55951-7e2e-4ddd-93e1-edaba145f932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:53.725888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee55951-7e2e-4ddd-93e1-edaba145f932 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:53.742681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 929/1861] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:57:53.744891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f6fd3-f515-4fdc-9954-c3265af8769c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:53.757703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f6fd3-f515-4fdc-9954-c3265af8769c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:53.763192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c9bac-3c2d-4638-aa4b-ae3be33fe894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:53.763810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c9bac-3c2d-4638-aa4b-ae3be33fe894 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:53.764545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c9bac-3c2d-4638-aa4b-ae3be33fe894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:53.764545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c9bac-3c2d-4638-aa4b-ae3be33fe894 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:53.764997 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c9bac-3c2d-4638-aa4b-ae3be33fe894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:53.766999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c9bac-3c2d-4638-aa4b-ae3be33fe894 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:53.800842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 933/1862] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:53 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:53.804573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee55951-7e2e-4ddd-93e1-edaba145f932 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:53.817085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee55951-7e2e-4ddd-93e1-edaba145f932 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:53.906009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2cbdd-3f92-47e2-a9a4-bb37d73d3460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:53.906738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2cbdd-3f92-47e2-a9a4-bb37d73d3460 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:53.907029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2cbdd-3f92-47e2-a9a4-bb37d73d3460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:53.907306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2cbdd-3f92-47e2-a9a4-bb37d73d3460 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:53.908009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2cbdd-3f92-47e2-a9a4-bb37d73d3460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:53.910235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2cbdd-3f92-47e2-a9a4-bb37d73d3460 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:53.947135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c9bac-3c2d-4638-aa4b-ae3be33fe894 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:53.954139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 930/1863] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:53 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:53.961039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c9bac-3c2d-4638-aa4b-ae3be33fe894 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:53.968026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326b9b-1c92-4bab-84ee-f59e4ae30440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:53.969033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326b9b-1c92-4bab-84ee-f59e4ae30440 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:53.969332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326b9b-1c92-4bab-84ee-f59e4ae30440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:53.969609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326b9b-1c92-4bab-84ee-f59e4ae30440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:53.970521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4326b9b-1c92-4bab-84ee-f59e4ae30440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:53.972856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4326b9b-1c92-4bab-84ee-f59e4ae30440 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:53.991004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 934/1864] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:53.992108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2cbdd-3f92-47e2-a9a4-bb37d73d3460 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:54.009786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2cbdd-3f92-47e2-a9a4-bb37d73d3460 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:54.020029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840934b-d1f6-4bfb-9041-34c431630cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:54.021018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840934b-d1f6-4bfb-9041-34c431630cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:54.021332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840934b-d1f6-4bfb-9041-34c431630cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:54.021602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840934b-d1f6-4bfb-9041-34c431630cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:54.022445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840934b-d1f6-4bfb-9041-34c431630cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:54.024369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840934b-d1f6-4bfb-9041-34c431630cbd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:54.056160 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 931/1865] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:57:54.058093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326b9b-1c92-4bab-84ee-f59e4ae30440 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:54.069883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326b9b-1c92-4bab-84ee-f59e4ae30440 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:54.075714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5a54-503a-4eb7-bb69-b4bc98614cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:54.076437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5a54-503a-4eb7-bb69-b4bc98614cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:54.076786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5a54-503a-4eb7-bb69-b4bc98614cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:54.077452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5a54-503a-4eb7-bb69-b4bc98614cbe admin admin] PATH_INFO: `/v3/projects/3a67ac193a604f2aaf2500e542116347/users/c3019ed6391747629a0c2315a9f9a711/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:54.080290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:54.102207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5a54-503a-4eb7-bb69-b4bc98614cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a67ac193a604f2aaf2500e542116347, user_id=c3019ed6391747629a0c2315a9f9a711, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:54.105979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5a54-503a-4eb7-bb69-b4bc98614cbe admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:54.126728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 935/1866] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:54 2022] PUT /identity/v3/projects/3a67ac193a604f2aaf2500e542116347/users/c3019ed6391747629a0c2315a9f9a711/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:54.137996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841f25b-b82c-49a4-ae7e-503354e81bd6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:54.138758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841f25b-b82c-49a4-ae7e-503354e81bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:54.139228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841f25b-b82c-49a4-ae7e-503354e81bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:54.183446 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2841f25b-b82c-49a4-ae7e-503354e81bd6 None None] MFA Rules not processed for user `c3019ed6391747629a0c2315a9f9a711`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:54.185026 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2841f25b-b82c-49a4-ae7e-503354e81bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:54.294601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5a54-503a-4eb7-bb69-b4bc98614cbe admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:54.296874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 932/1867] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:54.374673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5a54-503a-4eb7-bb69-b4bc98614cbe admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:54.379661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148f298-9a4b-4d16-b916-92850a69f6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:54.381255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148f298-9a4b-4d16-b916-92850a69f6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:54.381536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148f298-9a4b-4d16-b916-92850a69f6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:54.381822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148f298-9a4b-4d16-b916-92850a69f6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:54.382445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148f298-9a4b-4d16-b916-92850a69f6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:54.384419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148f298-9a4b-4d16-b916-92850a69f6a3 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:54.414131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 936/1868] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:54 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:54.418031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841f25b-b82c-49a4-ae7e-503354e81bd6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:54.430485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841f25b-b82c-49a4-ae7e-503354e81bd6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:54.464767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:54.515385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2eff2-8c8a-423e-8af7-eb926fe7a981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:54.516122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2eff2-8c8a-423e-8af7-eb926fe7a981 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:54.516407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2eff2-8c8a-423e-8af7-eb926fe7a981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:54.516685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2eff2-8c8a-423e-8af7-eb926fe7a981 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:54.517753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2eff2-8c8a-423e-8af7-eb926fe7a981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:54.519768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2eff2-8c8a-423e-8af7-eb926fe7a981 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:54.565903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 933/1869] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:54 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:54.570275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148f298-9a4b-4d16-b916-92850a69f6a3 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:54.586031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148f298-9a4b-4d16-b916-92850a69f6a3 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:54.593202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5199c8a-fe59-46c6-b54a-709979d5da18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:54.593202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5199c8a-fe59-46c6-b54a-709979d5da18 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:54.593202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5199c8a-fe59-46c6-b54a-709979d5da18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:54.595707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5199c8a-fe59-46c6-b54a-709979d5da18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:54.595707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5199c8a-fe59-46c6-b54a-709979d5da18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:54.596209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5199c8a-fe59-46c6-b54a-709979d5da18 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:54.609547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 937/1870] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:57:54.613316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2eff2-8c8a-423e-8af7-eb926fe7a981 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:54.624623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2eff2-8c8a-423e-8af7-eb926fe7a981 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:54.631054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1681e-6165-4583-9957-acc9cef392d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:54.631231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1681e-6165-4583-9957-acc9cef392d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:54.631697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1681e-6165-4583-9957-acc9cef392d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:54.632089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1681e-6165-4583-9957-acc9cef392d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:54.632676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1681e-6165-4583-9957-acc9cef392d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:54.634377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1681e-6165-4583-9957-acc9cef392d9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:54.645070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 934/1871] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:54.650625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5199c8a-fe59-46c6-b54a-709979d5da18 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:54.660680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5199c8a-fe59-46c6-b54a-709979d5da18 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:54.667335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f78e92-cc79-4283-868c-f615571c24d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:54.668020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f78e92-cc79-4283-868c-f615571c24d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:54.668280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f78e92-cc79-4283-868c-f615571c24d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:54.668551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f78e92-cc79-4283-868c-f615571c24d5 admin admin] PATH_INFO: `/v3/projects/cbe706371bed4efc8b0ea523bc9f4523/users/836f881962c74f11a24a5b39e091e021/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:54.686208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f78e92-cc79-4283-868c-f615571c24d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe706371bed4efc8b0ea523bc9f4523, user_id=836f881962c74f11a24a5b39e091e021, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:54.688934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f78e92-cc79-4283-868c-f615571c24d5 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:54.709274 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 938/1872] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:54 2022] PUT /identity/v3/projects/cbe706371bed4efc8b0ea523bc9f4523/users/836f881962c74f11a24a5b39e091e021/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:54.720082 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759e27-c25b-41f4-92c1-fd41f67bd544 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:54.720684 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759e27-c25b-41f4-92c1-fd41f67bd544 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:54.721053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759e27-c25b-41f4-92c1-fd41f67bd544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:54.758606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d7759e27-c25b-41f4-92c1-fd41f67bd544 None None] MFA Rules not processed for user `836f881962c74f11a24a5b39e091e021`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:54.760208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d7759e27-c25b-41f4-92c1-fd41f67bd544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:54.892265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 935/1873] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:54.898494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f78e92-cc79-4283-868c-f615571c24d5 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:55.033235 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f78e92-cc79-4283-868c-f615571c24d5 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:55.038691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f78b19-516e-41db-8e9c-9da0ba6ab901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:55.039728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f78b19-516e-41db-8e9c-9da0ba6ab901 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:55.040559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f78b19-516e-41db-8e9c-9da0ba6ab901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:55.040559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f78b19-516e-41db-8e9c-9da0ba6ab901 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:55.041261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f78b19-516e-41db-8e9c-9da0ba6ab901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:55.045650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f78b19-516e-41db-8e9c-9da0ba6ab901 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:55.065255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7759e27-c25b-41f4-92c1-fd41f67bd544 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:55.067835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d7759e27-c25b-41f4-92c1-fd41f67bd544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:55.095267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7759e27-c25b-41f4-92c1-fd41f67bd544 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:55.108450 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:55.109145 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:55.109354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 939/1874] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:54 2022] POST /identity/v3/projects => generated 377 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:57:55.109746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:55.109854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] PATH_INFO: `/v3/users/8749cb6226ad4604bc59b0eac41c8098` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:55.111483 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f78b19-516e-41db-8e9c-9da0ba6ab901 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:55.115231 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8749cb6226ad4604bc59b0eac41c8098)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:55.120754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:55.170963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:55.243597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8749cb6226ad4604bc59b0eac41c8098'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:55.274112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8749cb6226ad4604bc59b0eac41c8098'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:55.274922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8749cb6226ad4604bc59b0eac41c8098'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:55.285031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8749cb6226ad4604bc59b0eac41c8098'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:55.286449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-57558a96-8699-477b-8d10-8c77bf07cabe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8749cb6226ad4604bc59b0eac41c8098'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:55.291563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f78b19-516e-41db-8e9c-9da0ba6ab901 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:55.297176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c8eea-1ec2-462f-bd74-6b8bff60fd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:55.298405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c8eea-1ec2-462f-bd74-6b8bff60fd1b admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:55.298759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c8eea-1ec2-462f-bd74-6b8bff60fd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:55.299140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c8eea-1ec2-462f-bd74-6b8bff60fd1b admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:55.299941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c8eea-1ec2-462f-bd74-6b8bff60fd1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:55.302925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c8eea-1ec2-462f-bd74-6b8bff60fd1b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:55.332631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 936/1875] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:57:55 2022] DELETE /identity/v3/users/8749cb6226ad4604bc59b0eac41c8098 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:57:55.334973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b7d91-9348-41e8-ad87-e0be226f1f63 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:55.335264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b7d91-9348-41e8-ad87-e0be226f1f63 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:55.335612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b7d91-9348-41e8-ad87-e0be226f1f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:55.370370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 940/1876] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:55 2022] POST /identity/v3/users => generated 295 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:55.372887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c8eea-1ec2-462f-bd74-6b8bff60fd1b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:55.376616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-439c8eea-1ec2-462f-bd74-6b8bff60fd1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:55.382695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-de0b7d91-9348-41e8-ad87-e0be226f1f63 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:55.388044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-de0b7d91-9348-41e8-ad87-e0be226f1f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:55.401903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c8eea-1ec2-462f-bd74-6b8bff60fd1b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:55.407921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:55.589494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 937/1877] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:57:55.602231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dee3b8-9d36-4117-b45f-10e41a70a4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:55.606434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dee3b8-9d36-4117-b45f-10e41a70a4ee admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:55.606743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dee3b8-9d36-4117-b45f-10e41a70a4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:55.607031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dee3b8-9d36-4117-b45f-10e41a70a4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:55.614052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dee3b8-9d36-4117-b45f-10e41a70a4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:55.615656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dee3b8-9d36-4117-b45f-10e41a70a4ee admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:55.641242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 941/1878] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:57:55.647080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b7d91-9348-41e8-ad87-e0be226f1f63 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:55.664333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b7d91-9348-41e8-ad87-e0be226f1f63 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:55.671976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94f323-ae0d-46bb-a7a6-d2824c3e790c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:55.672710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94f323-ae0d-46bb-a7a6-d2824c3e790c admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:55.673107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94f323-ae0d-46bb-a7a6-d2824c3e790c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:55.673386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94f323-ae0d-46bb-a7a6-d2824c3e790c admin admin] PATH_INFO: `/v3/projects/f860ad33172c46719049754bd174818f/users/b8537d7a48224120b7abc5ec2c587680/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:55.694090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94f323-ae0d-46bb-a7a6-d2824c3e790c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f860ad33172c46719049754bd174818f, user_id=b8537d7a48224120b7abc5ec2c587680, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:55.697529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94f323-ae0d-46bb-a7a6-d2824c3e790c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:55.713782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dee3b8-9d36-4117-b45f-10e41a70a4ee admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:55.715590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e5dee3b8-9d36-4117-b45f-10e41a70a4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:55.722352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 938/1879] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:55 2022] PUT /identity/v3/projects/f860ad33172c46719049754bd174818f/users/b8537d7a48224120b7abc5ec2c587680/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:55.740771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94f323-ae0d-46bb-a7a6-d2824c3e790c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:55.851838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94f323-ae0d-46bb-a7a6-d2824c3e790c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:55.857877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea09c45-542c-492f-8ee4-ca9b685f157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:55.858981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09c45-542c-492f-8ee4-ca9b685f157c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:55.859421 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09c45-542c-492f-8ee4-ca9b685f157c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:55.859851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09c45-542c-492f-8ee4-ca9b685f157c admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:55.860840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea09c45-542c-492f-8ee4-ca9b685f157c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:55.863062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea09c45-542c-492f-8ee4-ca9b685f157c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:55.868308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dee3b8-9d36-4117-b45f-10e41a70a4ee admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:55.900192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 939/1880] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:55.905685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea09c45-542c-492f-8ee4-ca9b685f157c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:55.928405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea09c45-542c-492f-8ee4-ca9b685f157c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:55.944483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbd93b-fbaa-456c-add1-3b9a13d156af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:55.945500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbd93b-fbaa-456c-add1-3b9a13d156af admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:55.946059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbd93b-fbaa-456c-add1-3b9a13d156af admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:55.946489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbd93b-fbaa-456c-add1-3b9a13d156af admin admin] PATH_INFO: `/v3/projects/f860ad33172c46719049754bd174818f/users/b8537d7a48224120b7abc5ec2c587680/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:55.951225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbd93b-fbaa-456c-add1-3b9a13d156af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f860ad33172c46719049754bd174818f, user_id=b8537d7a48224120b7abc5ec2c587680, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:55.965446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62e711-6782-464c-8326-b00cd7648ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:55.966236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62e711-6782-464c-8326-b00cd7648ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:55.966607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62e711-6782-464c-8326-b00cd7648ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:55.966892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62e711-6782-464c-8326-b00cd7648ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:55.976719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbd93b-fbaa-456c-add1-3b9a13d156af admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:55.981838 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62e711-6782-464c-8326-b00cd7648ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:55.984118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62e711-6782-464c-8326-b00cd7648ec5 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:56.001678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 942/1881] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:57:56.009607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 940/1882] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:55 2022] PUT /identity/v3/projects/f860ad33172c46719049754bd174818f/users/b8537d7a48224120b7abc5ec2c587680/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:56.021400 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9630ad-88e6-40fb-8a74-49888bdda7b3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:56.022310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9630ad-88e6-40fb-8a74-49888bdda7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:56.022310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9630ad-88e6-40fb-8a74-49888bdda7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:56.031080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:56.074146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-fa9630ad-88e6-40fb-8a74-49888bdda7b3 None None] MFA Rules not processed for user `b8537d7a48224120b7abc5ec2c587680`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:56.076094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fa9630ad-88e6-40fb-8a74-49888bdda7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:56.217599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 943/1883] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:56.227251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbd93b-fbaa-456c-add1-3b9a13d156af admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:56.229774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7dcbd93b-fbaa-456c-add1-3b9a13d156af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:56.381459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbd93b-fbaa-456c-add1-3b9a13d156af admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:56.392960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02f40f-14d2-429c-a49c-53b8e443b107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:56.393668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02f40f-14d2-429c-a49c-53b8e443b107 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:56.393958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02f40f-14d2-429c-a49c-53b8e443b107 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:56.394235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02f40f-14d2-429c-a49c-53b8e443b107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:56.415590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3b02f40f-14d2-429c-a49c-53b8e443b107 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:56.476724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:56.524328 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02f40f-14d2-429c-a49c-53b8e443b107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:56.526735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02f40f-14d2-429c-a49c-53b8e443b107 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:56.540465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 941/1884] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:57:56.925380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9630ad-88e6-40fb-8a74-49888bdda7b3 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:56.928577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fa9630ad-88e6-40fb-8a74-49888bdda7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:57.058115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9630ad-88e6-40fb-8a74-49888bdda7b3 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:57.063976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:57.064821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:57.065176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:57.065521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] PATH_INFO: `/v3/projects/61b09f1eda6b4e698fb359d4951674d9` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:57.066862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b09f1eda6b4e698fb359d4951674d9)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:57.070193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:57.219544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b09f1eda6b4e698fb359d4951674d9', 'request_id': 'req-25ad52ab-05e4-42ee-8164-9a0728438429'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.227576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b09f1eda6b4e698fb359d4951674d9', 'request_id': 'req-25ad52ab-05e4-42ee-8164-9a0728438429'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.238311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b09f1eda6b4e698fb359d4951674d9', 'request_id': 'req-25ad52ab-05e4-42ee-8164-9a0728438429'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.254429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] The token cache is being invalidate because project 61b09f1eda6b4e698fb359d4951674d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:57:57.254900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.256655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.259364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 944/1885] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:57:56 2022] DELETE /identity/v3/projects/61b09f1eda6b4e698fb359d4951674d9 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:57.263465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02f40f-14d2-429c-a49c-53b8e443b107 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:57.276527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02f40f-14d2-429c-a49c-53b8e443b107 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:57.406749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:57.407432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:57.407724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:57.408002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] PATH_INFO: `/v3/users/33ddbc8eb7ac4e6ca216a310ff7e4d93` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:57.410299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ddbc8eb7ac4e6ca216a310ff7e4d93)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:57.413380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:57.509692 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ddbc8eb7ac4e6ca216a310ff7e4d93'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.522898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ddbc8eb7ac4e6ca216a310ff7e4d93'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.524334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ddbc8eb7ac4e6ca216a310ff7e4d93'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.535475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ddbc8eb7ac4e6ca216a310ff7e4d93'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.537295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ddbc8eb7ac4e6ca216a310ff7e4d93'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:57.563404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 942/1886] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:57:57 2022] DELETE /identity/v3/users/33ddbc8eb7ac4e6ca216a310ff7e4d93 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:58.227039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:58.228511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:58.246345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad52ab-05e4-42ee-8164-9a0728438429 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:58.283223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:58.365406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:58.366336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:58.367515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:58.368001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] PATH_INFO: `/v3/projects/0ce7fdd6f42d40248d4052167b0ee39f` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:58.369561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce7fdd6f42d40248d4052167b0ee39f)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:58.371850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:58.397245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:58.398670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:58.427684 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce7fdd6f42d40248d4052167b0ee39f', 'request_id': 'req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:58.434203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce7fdd6f42d40248d4052167b0ee39f', 'request_id': 'req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:58.443638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce7fdd6f42d40248d4052167b0ee39f', 'request_id': 'req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:58.458100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] The token cache is being invalidate because project 0ce7fdd6f42d40248d4052167b0ee39f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:57:58.458613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:58.459413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39fbfdf5-c616-408a-ae09-30c40b7fa2c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:57:58.459777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:58.461107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 945/1887] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:57:58 2022] DELETE /identity/v3/projects/0ce7fdd6f42d40248d4052167b0ee39f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:58.464633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149b249-85f9-4a2d-8c84-88a789f57319 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:58.465858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ad084-91b1-4a0e-9777-be96f4ca8c0c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:58.466207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ad084-91b1-4a0e-9777-be96f4ca8c0c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:58.466632 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ad084-91b1-4a0e-9777-be96f4ca8c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:58.496379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2e9ad084-91b1-4a0e-9777-be96f4ca8c0c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:58.497830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2e9ad084-91b1-4a0e-9777-be96f4ca8c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:58.524675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c9695-947e-4743-b8fe-88fd3e1d9cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:58.526229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c9695-947e-4743-b8fe-88fd3e1d9cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:58.526689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c9695-947e-4743-b8fe-88fd3e1d9cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:58.527072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c9695-947e-4743-b8fe-88fd3e1d9cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:58.529590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-773c9695-947e-4743-b8fe-88fd3e1d9cc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:58.617634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 946/1888] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:57:58.617865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c9695-947e-4743-b8fe-88fd3e1d9cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:58.618885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c9695-947e-4743-b8fe-88fd3e1d9cc5 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:58.619625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165254e3-5f07-4543-90aa-b9582395f665 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:58.620023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165254e3-5f07-4543-90aa-b9582395f665 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:58.620401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165254e3-5f07-4543-90aa-b9582395f665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:58.637835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 943/1889] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:57:58.652029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-165254e3-5f07-4543-90aa-b9582395f665 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:58.664573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-165254e3-5f07-4543-90aa-b9582395f665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:58.698816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 947/1890] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:57:58.702895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c9695-947e-4743-b8fe-88fd3e1d9cc5 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:58.721971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c9695-947e-4743-b8fe-88fd3e1d9cc5 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:58.737198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68a078-8378-462f-9308-a429e6ce6218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:58.738085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a078-8378-462f-9308-a429e6ce6218 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:58.738547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a078-8378-462f-9308-a429e6ce6218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:58.739557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a078-8378-462f-9308-a429e6ce6218 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:58.740365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68a078-8378-462f-9308-a429e6ce6218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:57:58.740933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68a078-8378-462f-9308-a429e6ce6218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:58.743259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68a078-8378-462f-9308-a429e6ce6218 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:58.774879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 944/1891] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:57:58.778945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165254e3-5f07-4543-90aa-b9582395f665 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:58.795211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165254e3-5f07-4543-90aa-b9582395f665 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:58.807931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631bc6b-2459-4859-84b4-94900063a67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:58.808969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631bc6b-2459-4859-84b4-94900063a67d admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:58.810214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631bc6b-2459-4859-84b4-94900063a67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:58.810680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631bc6b-2459-4859-84b4-94900063a67d admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:58.811527 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631bc6b-2459-4859-84b4-94900063a67d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:58.814548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631bc6b-2459-4859-84b4-94900063a67d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:58.855495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 948/1892] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:58 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:58.857270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68a078-8378-462f-9308-a429e6ce6218 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:58.874676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68a078-8378-462f-9308-a429e6ce6218 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:58.964034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668cdfe-f48e-4945-b60c-cee942b2e624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:58.964774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668cdfe-f48e-4945-b60c-cee942b2e624 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:58.965171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668cdfe-f48e-4945-b60c-cee942b2e624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:58.965551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668cdfe-f48e-4945-b60c-cee942b2e624 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:58.966300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668cdfe-f48e-4945-b60c-cee942b2e624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:58.968393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668cdfe-f48e-4945-b60c-cee942b2e624 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:58.993910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 945/1893] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:58 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:58.996893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631bc6b-2459-4859-84b4-94900063a67d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.011586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631bc6b-2459-4859-84b4-94900063a67d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.018838 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b5c79-1d6a-4b45-9bb3-e2916b68aa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.019742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b5c79-1d6a-4b45-9bb3-e2916b68aa9e admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.020152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b5c79-1d6a-4b45-9bb3-e2916b68aa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.020540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b5c79-1d6a-4b45-9bb3-e2916b68aa9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.021539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b5c79-1d6a-4b45-9bb3-e2916b68aa9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.023811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b5c79-1d6a-4b45-9bb3-e2916b68aa9e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.039977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 949/1894] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:59.044933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668cdfe-f48e-4945-b60c-cee942b2e624 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.056612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668cdfe-f48e-4945-b60c-cee942b2e624 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.061318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5005f-64fa-48fc-a60e-4f5429df229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.061921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5005f-64fa-48fc-a60e-4f5429df229c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.062198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5005f-64fa-48fc-a60e-4f5429df229c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.062466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5005f-64fa-48fc-a60e-4f5429df229c admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.063101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5005f-64fa-48fc-a60e-4f5429df229c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.064740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5005f-64fa-48fc-a60e-4f5429df229c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.077428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 946/1895] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:57:59.082079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b5c79-1d6a-4b45-9bb3-e2916b68aa9e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.092766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b5c79-1d6a-4b45-9bb3-e2916b68aa9e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.097921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2c758-e5d1-4916-9277-cc0ee9443c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.098706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2c758-e5d1-4916-9277-cc0ee9443c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.099228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2c758-e5d1-4916-9277-cc0ee9443c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.099649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2c758-e5d1-4916-9277-cc0ee9443c76 admin admin] PATH_INFO: `/v3/projects/be0a14d8a49c456090394fd0ac5c988a/users/7dc18df1cb1d45a3a24be6e4f813c575/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.103889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:59.122977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2c758-e5d1-4916-9277-cc0ee9443c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0a14d8a49c456090394fd0ac5c988a, user_id=7dc18df1cb1d45a3a24be6e4f813c575, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.127763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2c758-e5d1-4916-9277-cc0ee9443c76 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.144756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 950/1896] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:59 2022] PUT /identity/v3/projects/be0a14d8a49c456090394fd0ac5c988a/users/7dc18df1cb1d45a3a24be6e4f813c575/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:59.155655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b444a-00b8-48a3-9d2f-e5acd4ae01c1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.156114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b444a-00b8-48a3-9d2f-e5acd4ae01c1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.156685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b444a-00b8-48a3-9d2f-e5acd4ae01c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.190777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b41b444a-00b8-48a3-9d2f-e5acd4ae01c1 None None] MFA Rules not processed for user `7dc18df1cb1d45a3a24be6e4f813c575`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:59.193260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b41b444a-00b8-48a3-9d2f-e5acd4ae01c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:59.207762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:59.295852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 947/1897] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:57:59.300525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2c758-e5d1-4916-9277-cc0ee9443c76 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.375216 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2c758-e5d1-4916-9277-cc0ee9443c76 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.394012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c54f-9add-4aeb-9acf-6c1eba55fa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.394725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c54f-9add-4aeb-9acf-6c1eba55fa27 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.395236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c54f-9add-4aeb-9acf-6c1eba55fa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.395707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c54f-9add-4aeb-9acf-6c1eba55fa27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.396509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585c54f-9add-4aeb-9acf-6c1eba55fa27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.398801 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585c54f-9add-4aeb-9acf-6c1eba55fa27 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.412256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b444a-00b8-48a3-9d2f-e5acd4ae01c1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.414225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b41b444a-00b8-48a3-9d2f-e5acd4ae01c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:59.449459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 951/1898] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:57:59 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:59.450350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c54f-9add-4aeb-9acf-6c1eba55fa27 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.507745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b444a-00b8-48a3-9d2f-e5acd4ae01c1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.533619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c54f-9add-4aeb-9acf-6c1eba55fa27 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.547884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942846d9-8c40-4aa2-8ed1-c3bef48f89db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.548536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942846d9-8c40-4aa2-8ed1-c3bef48f89db admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.548833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942846d9-8c40-4aa2-8ed1-c3bef48f89db admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.549105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942846d9-8c40-4aa2-8ed1-c3bef48f89db admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.549766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942846d9-8c40-4aa2-8ed1-c3bef48f89db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.552683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942846d9-8c40-4aa2-8ed1-c3bef48f89db admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.586422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 952/1899] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:57:59 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:57:59.586619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c684c3d-303b-44ab-b10b-97e524ea03b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.586899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c684c3d-303b-44ab-b10b-97e524ea03b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.587068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c684c3d-303b-44ab-b10b-97e524ea03b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.587366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c684c3d-303b-44ab-b10b-97e524ea03b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.589267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942846d9-8c40-4aa2-8ed1-c3bef48f89db admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.594602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8c684c3d-303b-44ab-b10b-97e524ea03b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:57:59.600630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942846d9-8c40-4aa2-8ed1-c3bef48f89db admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.606553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524574f-327f-4216-bad5-764adb284c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.607188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524574f-327f-4216-bad5-764adb284c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.607459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524574f-327f-4216-bad5-764adb284c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.607728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524574f-327f-4216-bad5-764adb284c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.608671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524574f-327f-4216-bad5-764adb284c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.610126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524574f-327f-4216-bad5-764adb284c94 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.626491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 953/1900] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:57:59.631862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524574f-327f-4216-bad5-764adb284c94 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.644752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524574f-327f-4216-bad5-764adb284c94 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.656502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0e651-b894-4e96-8905-aaf8807abee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.657269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0e651-b894-4e96-8905-aaf8807abee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.657552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0e651-b894-4e96-8905-aaf8807abee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.657825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0e651-b894-4e96-8905-aaf8807abee4 admin admin] PATH_INFO: `/v3/projects/7750efdd02d94d3c9b092c7c185b01cc/users/0296a52c267843ad9109a28cafd3185d/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.678608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0e651-b894-4e96-8905-aaf8807abee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7750efdd02d94d3c9b092c7c185b01cc, user_id=0296a52c267843ad9109a28cafd3185d, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.680262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c684c3d-303b-44ab-b10b-97e524ea03b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.682005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0e651-b894-4e96-8905-aaf8807abee4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.682297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c684c3d-303b-44ab-b10b-97e524ea03b8 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.696755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 948/1901] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:57:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:57:59.706363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 954/1902] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:59 2022] PUT /identity/v3/projects/7750efdd02d94d3c9b092c7c185b01cc/users/0296a52c267843ad9109a28cafd3185d/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:57:59.711232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c684c3d-303b-44ab-b10b-97e524ea03b8 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.722737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c684c3d-303b-44ab-b10b-97e524ea03b8 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.745064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:57:59.813607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a4d03-cd4e-4d83-883b-e6fc0ab48e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.816750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a4d03-cd4e-4d83-883b-e6fc0ab48e8d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.817451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a4d03-cd4e-4d83-883b-e6fc0ab48e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.817694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a4d03-cd4e-4d83-883b-e6fc0ab48e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.818594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a4d03-cd4e-4d83-883b-e6fc0ab48e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.820210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a4d03-cd4e-4d83-883b-e6fc0ab48e8d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.845210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 949/1903] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:57:59.846804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0e651-b894-4e96-8905-aaf8807abee4 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:57:59.862519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0e651-b894-4e96-8905-aaf8807abee4 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:57:59.868648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3fe8a-ad9e-4d83-b072-d05d1c753884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:57:59.869355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3fe8a-ad9e-4d83-b072-d05d1c753884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.869633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3fe8a-ad9e-4d83-b072-d05d1c753884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.869901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3fe8a-ad9e-4d83-b072-d05d1c753884 admin admin] PATH_INFO: `/v3/projects/7750efdd02d94d3c9b092c7c185b01cc/users/0296a52c267843ad9109a28cafd3185d/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.871815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3fe8a-ad9e-4d83-b072-d05d1c753884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7750efdd02d94d3c9b092c7c185b01cc, user_id=0296a52c267843ad9109a28cafd3185d, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:57:59.874527 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3fe8a-ad9e-4d83-b072-d05d1c753884 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:57:59.888212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 955/1904] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:57:59 2022] PUT /identity/v3/projects/7750efdd02d94d3c9b092c7c185b01cc/users/0296a52c267843ad9109a28cafd3185d/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:57:59.899482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6a949-2237-45d3-8a47-7f5815dbf387 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:57:59.900003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6a949-2237-45d3-8a47-7f5815dbf387 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:57:59.900357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6a949-2237-45d3-8a47-7f5815dbf387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:57:59.934660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-45d6a949-2237-45d3-8a47-7f5815dbf387 None None] MFA Rules not processed for user `0296a52c267843ad9109a28cafd3185d`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:57:59.936021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-45d6a949-2237-45d3-8a47-7f5815dbf387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:00.054061 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 950/1905] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:57:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:00.065657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3fe8a-ad9e-4d83-b072-d05d1c753884 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:00.088544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:00.161376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3fe8a-ad9e-4d83-b072-d05d1c753884 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:00.167067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ed673-93b3-431d-af57-6a4e4e842789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:00.168176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ed673-93b3-431d-af57-6a4e4e842789 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:00.168536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ed673-93b3-431d-af57-6a4e4e842789 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:00.168849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ed673-93b3-431d-af57-6a4e4e842789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:00.181814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ed673-93b3-431d-af57-6a4e4e842789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:00.183382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ed673-93b3-431d-af57-6a4e4e842789 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:00.193769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 956/1906] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:00.226351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6a949-2237-45d3-8a47-7f5815dbf387 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:00.301075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6a949-2237-45d3-8a47-7f5815dbf387 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:00.307057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:00.307719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:00.308013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:00.308297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] PATH_INFO: `/v3/users/7dc18df1cb1d45a3a24be6e4f813c575` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:00.309772 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc18df1cb1d45a3a24be6e4f813c575)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:00.311731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:00.366732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:00.368743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc18df1cb1d45a3a24be6e4f813c575'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:00.378526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc18df1cb1d45a3a24be6e4f813c575'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:00.379176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc18df1cb1d45a3a24be6e4f813c575'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:00.387163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc18df1cb1d45a3a24be6e4f813c575'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:00.387869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc18df1cb1d45a3a24be6e4f813c575'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:00.407693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 951/1907] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:00 2022] DELETE /identity/v3/users/7dc18df1cb1d45a3a24be6e4f813c575 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:00.412152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ce53-d0b9-4e8c-b4b8-7997a11e9d00 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:00.412644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ce53-d0b9-4e8c-b4b8-7997a11e9d00 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:00.412725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ce53-d0b9-4e8c-b4b8-7997a11e9d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:00.439979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-0c14ce53-d0b9-4e8c-b4b8-7997a11e9d00 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:00.441368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0c14ce53-d0b9-4e8c-b4b8-7997a11e9d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:00.508631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 957/1908] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:00.518284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:00.519915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:00.623327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d242c2-b651-49f6-901c-e3e605ee9281 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:00.627588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e54cc7-49eb-40e5-a78f-168f45f87a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:00.628214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e54cc7-49eb-40e5-a78f-168f45f87a45 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:00.628594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e54cc7-49eb-40e5-a78f-168f45f87a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:00.628877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e54cc7-49eb-40e5-a78f-168f45f87a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:00.640192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e54cc7-49eb-40e5-a78f-168f45f87a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:00.641691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e54cc7-49eb-40e5-a78f-168f45f87a45 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:00.653718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 952/1909] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:01.370696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14ce53-d0b9-4e8c-b4b8-7997a11e9d00 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:01.372172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0c14ce53-d0b9-4e8c-b4b8-7997a11e9d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:01.386882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14ce53-d0b9-4e8c-b4b8-7997a11e9d00 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:01.392129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:01.392901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:01.393087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:01.393435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] PATH_INFO: `/v3/projects/be0a14d8a49c456090394fd0ac5c988a` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:01.394571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0a14d8a49c456090394fd0ac5c988a)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:01.398453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:01.477480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0a14d8a49c456090394fd0ac5c988a', 'request_id': 'req-c8bfb33e-30ca-4387-9684-9084efcb82c4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.487739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0a14d8a49c456090394fd0ac5c988a', 'request_id': 'req-c8bfb33e-30ca-4387-9684-9084efcb82c4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.501036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0a14d8a49c456090394fd0ac5c988a', 'request_id': 'req-c8bfb33e-30ca-4387-9684-9084efcb82c4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.522794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] The token cache is being invalidate because project be0a14d8a49c456090394fd0ac5c988a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:01.523268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.524133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.526142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 958/1910] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:01 2022] DELETE /identity/v3/projects/be0a14d8a49c456090394fd0ac5c988a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:01.530154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e54cc7-49eb-40e5-a78f-168f45f87a45 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:01.548418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e54cc7-49eb-40e5-a78f-168f45f87a45 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:01.621738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:01.622388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:01.622670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:01.622943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] PATH_INFO: `/v3/users/0296a52c267843ad9109a28cafd3185d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:01.624067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0296a52c267843ad9109a28cafd3185d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:01.626037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:01.675510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0296a52c267843ad9109a28cafd3185d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.684075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0296a52c267843ad9109a28cafd3185d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.685331 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0296a52c267843ad9109a28cafd3185d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.695433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0296a52c267843ad9109a28cafd3185d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.696182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f149c8b5-55d3-4bed-a0f4-63c7c46392d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0296a52c267843ad9109a28cafd3185d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:01.700240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:01.724580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 953/1911] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:01 2022] DELETE /identity/v3/users/0296a52c267843ad9109a28cafd3185d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:02.039582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:02.042733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:02.057428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfb33e-30ca-4387-9684-9084efcb82c4 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:02.093057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:02.128487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:02.129304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:02.131140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:02.131489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] PATH_INFO: `/v3/projects/7750efdd02d94d3c9b092c7c185b01cc` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:02.132714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7750efdd02d94d3c9b092c7c185b01cc)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:02.134991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:02.190946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7750efdd02d94d3c9b092c7c185b01cc', 'request_id': 'req-4020c33a-28c6-4885-b9b5-6728f46e2bd7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:02.198542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7750efdd02d94d3c9b092c7c185b01cc', 'request_id': 'req-4020c33a-28c6-4885-b9b5-6728f46e2bd7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:02.207198 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7750efdd02d94d3c9b092c7c185b01cc', 'request_id': 'req-4020c33a-28c6-4885-b9b5-6728f46e2bd7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:02.222405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] The token cache is being invalidate because project 7750efdd02d94d3c9b092c7c185b01cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:02.222763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:02.223579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4020c33a-28c6-4885-b9b5-6728f46e2bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:02.226655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 959/1912] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:02 2022] DELETE /identity/v3/projects/7750efdd02d94d3c9b092c7c185b01cc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:02.230431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4a5be-b6da-45c5-adc9-89c2910ea2e2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:02.230717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4a5be-b6da-45c5-adc9-89c2910ea2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:02.230991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4a5be-b6da-45c5-adc9-89c2910ea2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:02.252490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-52b4a5be-b6da-45c5-adc9-89c2910ea2e2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:02.253752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-52b4a5be-b6da-45c5-adc9-89c2910ea2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:02.330737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 954/1913] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:02.342469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaaeac-6f4f-4088-9ed6-961d9957db05 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:02.342605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaaeac-6f4f-4088-9ed6-961d9957db05 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:02.342975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaaeac-6f4f-4088-9ed6-961d9957db05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:02.376950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5adaaeac-6f4f-4088-9ed6-961d9957db05 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:02.378406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5adaaeac-6f4f-4088-9ed6-961d9957db05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:02.394677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 960/1914] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:02.406859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4a5be-b6da-45c5-adc9-89c2910ea2e2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:02.423413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4a5be-b6da-45c5-adc9-89c2910ea2e2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:02.432071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6bf8f-9010-490c-ba93-43768d87bc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:02.432688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6bf8f-9010-490c-ba93-43768d87bc23 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:02.433129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6bf8f-9010-490c-ba93-43768d87bc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:02.433459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6bf8f-9010-490c-ba93-43768d87bc23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:02.434107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6bf8f-9010-490c-ba93-43768d87bc23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:02.434416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6bf8f-9010-490c-ba93-43768d87bc23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:02.436031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6bf8f-9010-490c-ba93-43768d87bc23 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:02.456916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 955/1915] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:58:02.461792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaaeac-6f4f-4088-9ed6-961d9957db05 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:02.472823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaaeac-6f4f-4088-9ed6-961d9957db05 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:02.479057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f2aba-ac4a-4dc4-8cef-bb899a0b18c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:02.479762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f2aba-ac4a-4dc4-8cef-bb899a0b18c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:02.480209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f2aba-ac4a-4dc4-8cef-bb899a0b18c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:02.480959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f2aba-ac4a-4dc4-8cef-bb899a0b18c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:02.482160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f2aba-ac4a-4dc4-8cef-bb899a0b18c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:02.484644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f2aba-ac4a-4dc4-8cef-bb899a0b18c2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:02.548938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 961/1916] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:02 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:02.550974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6bf8f-9010-490c-ba93-43768d87bc23 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:02.578483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6bf8f-9010-490c-ba93-43768d87bc23 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:02.706359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dea3a-d59f-449d-ab5e-0982bfab7826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:02.707219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dea3a-d59f-449d-ab5e-0982bfab7826 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:02.709305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dea3a-d59f-449d-ab5e-0982bfab7826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:02.709900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dea3a-d59f-449d-ab5e-0982bfab7826 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:02.711949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dea3a-d59f-449d-ab5e-0982bfab7826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:02.724776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dea3a-d59f-449d-ab5e-0982bfab7826 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:02.779490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 956/1917] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:02 2022] POST /identity/v3/users => generated 294 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:02.784178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f2aba-ac4a-4dc4-8cef-bb899a0b18c2 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:02.804266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f2aba-ac4a-4dc4-8cef-bb899a0b18c2 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:02.818356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdbf71-ce3b-4c13-86c7-466a192e5793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:02.819461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdbf71-ce3b-4c13-86c7-466a192e5793 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:02.819845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdbf71-ce3b-4c13-86c7-466a192e5793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:02.820374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdbf71-ce3b-4c13-86c7-466a192e5793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:02.821631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdbf71-ce3b-4c13-86c7-466a192e5793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:02.825435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdbf71-ce3b-4c13-86c7-466a192e5793 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:02.851324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 962/1918] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:02.855379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dea3a-d59f-449d-ab5e-0982bfab7826 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:02.870059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dea3a-d59f-449d-ab5e-0982bfab7826 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:02.889055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529baa8-8d74-4e88-be1e-1793c4cdb93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:02.889818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529baa8-8d74-4e88-be1e-1793c4cdb93d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:02.890204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529baa8-8d74-4e88-be1e-1793c4cdb93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:02.891030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529baa8-8d74-4e88-be1e-1793c4cdb93d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:02.891791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529baa8-8d74-4e88-be1e-1793c4cdb93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:02.893861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529baa8-8d74-4e88-be1e-1793c4cdb93d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:02.907910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 957/1919] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:02.917240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdbf71-ce3b-4c13-86c7-466a192e5793 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:02.928724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdbf71-ce3b-4c13-86c7-466a192e5793 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:02.935037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616bb42-10b9-4272-85b1-3e0ab2a0e1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:02.936144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616bb42-10b9-4272-85b1-3e0ab2a0e1f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:02.936502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616bb42-10b9-4272-85b1-3e0ab2a0e1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:02.936885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616bb42-10b9-4272-85b1-3e0ab2a0e1f7 admin admin] PATH_INFO: `/v3/projects/5999c1eac5434e18b1bd6430663a41de/users/03251a7a44fa4f14ab3b5a2a8c3b6014/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:02.960401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616bb42-10b9-4272-85b1-3e0ab2a0e1f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5999c1eac5434e18b1bd6430663a41de, user_id=03251a7a44fa4f14ab3b5a2a8c3b6014, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:02.964004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616bb42-10b9-4272-85b1-3e0ab2a0e1f7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:02.992568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 963/1920] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:02 2022] PUT /identity/v3/projects/5999c1eac5434e18b1bd6430663a41de/users/03251a7a44fa4f14ab3b5a2a8c3b6014/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:03.002052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495517a8-3b91-47df-979a-2d19832fc3ce None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:03.002527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495517a8-3b91-47df-979a-2d19832fc3ce None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:03.003046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495517a8-3b91-47df-979a-2d19832fc3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:03.069691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-495517a8-3b91-47df-979a-2d19832fc3ce None None] MFA Rules not processed for user `03251a7a44fa4f14ab3b5a2a8c3b6014`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:03.071889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-495517a8-3b91-47df-979a-2d19832fc3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:03.123219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:03.167056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 958/1921] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:03.171584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616bb42-10b9-4272-85b1-3e0ab2a0e1f7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:03.243551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616bb42-10b9-4272-85b1-3e0ab2a0e1f7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:03.248249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a563cdb-8f46-4846-9d27-4004b28568a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:03.248885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a563cdb-8f46-4846-9d27-4004b28568a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:03.249166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a563cdb-8f46-4846-9d27-4004b28568a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:03.249445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a563cdb-8f46-4846-9d27-4004b28568a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:03.250076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a563cdb-8f46-4846-9d27-4004b28568a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:03.251999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a563cdb-8f46-4846-9d27-4004b28568a8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:03.265623 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:03.289110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 964/1922] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:03 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:03.293166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495517a8-3b91-47df-979a-2d19832fc3ce None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:03.304563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495517a8-3b91-47df-979a-2d19832fc3ce None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:03.367626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba6dcc-c990-49e3-979c-d2f9166249c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:03.368250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba6dcc-c990-49e3-979c-d2f9166249c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:03.368528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba6dcc-c990-49e3-979c-d2f9166249c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:03.368839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba6dcc-c990-49e3-979c-d2f9166249c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:03.369476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba6dcc-c990-49e3-979c-d2f9166249c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:03.371572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba6dcc-c990-49e3-979c-d2f9166249c9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:03.406625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 959/1923] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:03 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:03.406854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a563cdb-8f46-4846-9d27-4004b28568a8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:03.414241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a563cdb-8f46-4846-9d27-4004b28568a8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:03.419502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a963cf5-6b40-4b86-8186-f2e27067444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:03.420122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a963cf5-6b40-4b86-8186-f2e27067444b admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:03.420402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a963cf5-6b40-4b86-8186-f2e27067444b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:03.420673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a963cf5-6b40-4b86-8186-f2e27067444b admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:03.421343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a963cf5-6b40-4b86-8186-f2e27067444b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:03.423024 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a963cf5-6b40-4b86-8186-f2e27067444b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:03.437980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 965/1924] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:58:03.442572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba6dcc-c990-49e3-979c-d2f9166249c9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:03.450144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba6dcc-c990-49e3-979c-d2f9166249c9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:03.455717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf7c7c-3f42-4591-bf14-a7f7ba2a6600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:03.456498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7c7c-3f42-4591-bf14-a7f7ba2a6600 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:03.456928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7c7c-3f42-4591-bf14-a7f7ba2a6600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:03.457819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7c7c-3f42-4591-bf14-a7f7ba2a6600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:03.470530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf7c7c-3f42-4591-bf14-a7f7ba2a6600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:03.472277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf7c7c-3f42-4591-bf14-a7f7ba2a6600 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:03.495378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 960/1925] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:03.496054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a963cf5-6b40-4b86-8186-f2e27067444b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:03.515227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a963cf5-6b40-4b86-8186-f2e27067444b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:03.521793 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e126bc-e65e-43e9-a123-7144fc8b83a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:03.522468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e126bc-e65e-43e9-a123-7144fc8b83a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:03.522749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e126bc-e65e-43e9-a123-7144fc8b83a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:03.523024 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e126bc-e65e-43e9-a123-7144fc8b83a2 admin admin] PATH_INFO: `/v3/projects/39d5acde040b4c578930417d5e5f0841/users/2372e419b08742e88dd3e79d280c76c3/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:03.539611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e126bc-e65e-43e9-a123-7144fc8b83a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d5acde040b4c578930417d5e5f0841, user_id=2372e419b08742e88dd3e79d280c76c3, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:03.543360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e126bc-e65e-43e9-a123-7144fc8b83a2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:03.557830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 966/1926] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:03 2022] PUT /identity/v3/projects/39d5acde040b4c578930417d5e5f0841/users/2372e419b08742e88dd3e79d280c76c3/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:03.569135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886eafd8-27a1-460b-91d1-2b118a1e4220 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:03.569635 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886eafd8-27a1-460b-91d1-2b118a1e4220 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:03.569761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886eafd8-27a1-460b-91d1-2b118a1e4220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:03.599455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-886eafd8-27a1-460b-91d1-2b118a1e4220 None None] MFA Rules not processed for user `2372e419b08742e88dd3e79d280c76c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:03.600691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-886eafd8-27a1-460b-91d1-2b118a1e4220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:03.666812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 961/1927] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:03.671240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e126bc-e65e-43e9-a123-7144fc8b83a2 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:03.733171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e126bc-e65e-43e9-a123-7144fc8b83a2 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:03.737817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769b4d9-68ad-4550-ae81-05aeb0cec197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:03.738423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769b4d9-68ad-4550-ae81-05aeb0cec197 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:03.739514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769b4d9-68ad-4550-ae81-05aeb0cec197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:03.739791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769b4d9-68ad-4550-ae81-05aeb0cec197 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:03.740435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769b4d9-68ad-4550-ae81-05aeb0cec197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:03.742394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769b4d9-68ad-4550-ae81-05aeb0cec197 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:03.775741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 967/1928] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:03 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:03.776884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886eafd8-27a1-460b-91d1-2b118a1e4220 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:03.790478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886eafd8-27a1-460b-91d1-2b118a1e4220 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:03.861742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:03.929298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95489800-2778-4410-8e3a-1b29abeda45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:03.930156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95489800-2778-4410-8e3a-1b29abeda45f admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:03.930355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95489800-2778-4410-8e3a-1b29abeda45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:03.930798 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95489800-2778-4410-8e3a-1b29abeda45f admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:03.931454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95489800-2778-4410-8e3a-1b29abeda45f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:03.935819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95489800-2778-4410-8e3a-1b29abeda45f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:03.989475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 962/1929] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:03 2022] POST /identity/v3/users => generated 299 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:03.994504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769b4d9-68ad-4550-ae81-05aeb0cec197 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:04.018526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95489800-2778-4410-8e3a-1b29abeda45f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:04.021226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-95489800-2778-4410-8e3a-1b29abeda45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:04.024149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769b4d9-68ad-4550-ae81-05aeb0cec197 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:04.034655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00209e-07dc-4bf3-8ab1-773fe2cdd7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:04.036235 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00209e-07dc-4bf3-8ab1-773fe2cdd7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.036643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00209e-07dc-4bf3-8ab1-773fe2cdd7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.037104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00209e-07dc-4bf3-8ab1-773fe2cdd7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.043033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95489800-2778-4410-8e3a-1b29abeda45f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:04.043229 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00209e-07dc-4bf3-8ab1-773fe2cdd7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:04.044443 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00209e-07dc-4bf3-8ab1-773fe2cdd7fc admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:04.059728 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:04.060535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.060942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.061311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] PATH_INFO: `/v3/users/c3019ed6391747629a0c2315a9f9a711` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.062698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3019ed6391747629a0c2315a9f9a711)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:04.068976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 968/1930] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:04.074189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00209e-07dc-4bf3-8ab1-773fe2cdd7fc admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:04.080033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:04.125691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00209e-07dc-4bf3-8ab1-773fe2cdd7fc admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:04.136884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f004e-39a6-4706-b295-839cc789326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:04.137567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f004e-39a6-4706-b295-839cc789326f admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.137850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f004e-39a6-4706-b295-839cc789326f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.138123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f004e-39a6-4706-b295-839cc789326f admin admin] PATH_INFO: `/v3/projects/93c5056dda384cd1b38ea57641447e81/users/892d708a05fa429ba84213f9716c576a/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.179902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f004e-39a6-4706-b295-839cc789326f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c5056dda384cd1b38ea57641447e81, user_id=892d708a05fa429ba84213f9716c576a, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:04.193727 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f004e-39a6-4706-b295-839cc789326f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:04.221020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 969/1931] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:04 2022] PUT /identity/v3/projects/93c5056dda384cd1b38ea57641447e81/users/892d708a05fa429ba84213f9716c576a/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:04.224837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f004e-39a6-4706-b295-839cc789326f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:04.225261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3019ed6391747629a0c2315a9f9a711'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:04.262730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3019ed6391747629a0c2315a9f9a711'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:04.263580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3019ed6391747629a0c2315a9f9a711'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:04.279012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3019ed6391747629a0c2315a9f9a711'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:04.280461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c5cfcb09-3fbb-4c74-aa92-4ff993c37e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3019ed6391747629a0c2315a9f9a711'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:04.308563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f004e-39a6-4706-b295-839cc789326f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:04.313673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45b51c-e062-4d59-b1eb-4a36a6240e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:04.314294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45b51c-e062-4d59-b1eb-4a36a6240e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.314783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45b51c-e062-4d59-b1eb-4a36a6240e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.315309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45b51c-e062-4d59-b1eb-4a36a6240e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.316008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45b51c-e062-4d59-b1eb-4a36a6240e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:04.318654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45b51c-e062-4d59-b1eb-4a36a6240e35 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:04.328041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 963/1932] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:04 2022] DELETE /identity/v3/users/c3019ed6391747629a0c2315a9f9a711 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:04.332844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e461e-f55e-470b-8cef-1746c33005b6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.333440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e461e-f55e-470b-8cef-1746c33005b6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.333922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e461e-f55e-470b-8cef-1746c33005b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.335291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:04.351347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 970/1933] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:58:04.352271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45b51c-e062-4d59-b1eb-4a36a6240e35 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:04.354739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-da45b51c-e062-4d59-b1eb-4a36a6240e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:04.383924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45b51c-e062-4d59-b1eb-4a36a6240e35 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:04.385161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ab8e461e-f55e-470b-8cef-1746c33005b6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:04.386478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ab8e461e-f55e-470b-8cef-1746c33005b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:04.520660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 964/1934] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:04.536205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e461e-f55e-470b-8cef-1746c33005b6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:04.538670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d5553-135e-4d57-a7df-9f6c3aa9be01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:04.539389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d5553-135e-4d57-a7df-9f6c3aa9be01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.539676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d5553-135e-4d57-a7df-9f6c3aa9be01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.539963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d5553-135e-4d57-a7df-9f6c3aa9be01 admin admin] PATH_INFO: `/v3/projects/93c5056dda384cd1b38ea57641447e81/users/892d708a05fa429ba84213f9716c576a/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.541711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ab8e461e-f55e-470b-8cef-1746c33005b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:04.546673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d5553-135e-4d57-a7df-9f6c3aa9be01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c5056dda384cd1b38ea57641447e81, user_id=892d708a05fa429ba84213f9716c576a, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:04.551331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d5553-135e-4d57-a7df-9f6c3aa9be01 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:04.586748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 971/1935] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:04 2022] PUT /identity/v3/projects/93c5056dda384cd1b38ea57641447e81/users/892d708a05fa429ba84213f9716c576a/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:04.602659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7c536-84ff-4cb4-8004-6ffda937ff43 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.604179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7c536-84ff-4cb4-8004-6ffda937ff43 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.604475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7c536-84ff-4cb4-8004-6ffda937ff43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.639618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e461e-f55e-470b-8cef-1746c33005b6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:04.640981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:04.643008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9ba7c536-84ff-4cb4-8004-6ffda937ff43 None None] MFA Rules not processed for user `892d708a05fa429ba84213f9716c576a`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:04.645420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9ba7c536-84ff-4cb4-8004-6ffda937ff43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:04.705448 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879a141-e943-43d0-93c8-8ecb3c528131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:04.706078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879a141-e943-43d0-93c8-8ecb3c528131 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.706435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879a141-e943-43d0-93c8-8ecb3c528131 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.706785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879a141-e943-43d0-93c8-8ecb3c528131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.749387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 972/1936] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:04.778195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7c536-84ff-4cb4-8004-6ffda937ff43 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:04.781117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9ba7c536-84ff-4cb4-8004-6ffda937ff43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:04.808167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879a141-e943-43d0-93c8-8ecb3c528131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:04.810109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879a141-e943-43d0-93c8-8ecb3c528131 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:04.820526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 965/1937] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:04.851480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7c536-84ff-4cb4-8004-6ffda937ff43 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:04.855770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7a16e-0cfc-4ed9-8ed3-43107b96172a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:04.856643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7a16e-0cfc-4ed9-8ed3-43107b96172a service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.857015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7a16e-0cfc-4ed9-8ed3-43107b96172a service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.857687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7a16e-0cfc-4ed9-8ed3-43107b96172a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.868756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7a16e-0cfc-4ed9-8ed3-43107b96172a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:04.870113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7a16e-0cfc-4ed9-8ed3-43107b96172a service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:04.881031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 973/1938] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:04.979735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f225edb-1eff-4fa1-a99a-aa043f7f7d6e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.980705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f225edb-1eff-4fa1-a99a-aa043f7f7d6e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.981191 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f225edb-1eff-4fa1-a99a-aa043f7f7d6e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:04.982746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 966/1939] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 10:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:58:04.993983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf8489-b5cb-47b2-a9cf-2032bdee25b2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:04.994405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf8489-b5cb-47b2-a9cf-2032bdee25b2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:04.994923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf8489-b5cb-47b2-a9cf-2032bdee25b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:05.038308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e1cf8489-b5cb-47b2-a9cf-2032bdee25b2 None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:05.045900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e1cf8489-b5cb-47b2-a9cf-2032bdee25b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:05.103056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:05.189897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f225edb-1eff-4fa1-a99a-aa043f7f7d6e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:05.192384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5f225edb-1eff-4fa1-a99a-aa043f7f7d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:05.232416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f225edb-1eff-4fa1-a99a-aa043f7f7d6e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:05.250097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:05.250818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:05.251105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:05.251385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] PATH_INFO: `/v3/projects/3a67ac193a604f2aaf2500e542116347` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:05.263950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 974/1940] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:05.267002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a67ac193a604f2aaf2500e542116347)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:05.274224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:05.284298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf8489-b5cb-47b2-a9cf-2032bdee25b2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:05.287646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e1cf8489-b5cb-47b2-a9cf-2032bdee25b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:05.316070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf8489-b5cb-47b2-a9cf-2032bdee25b2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:05.329575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89d949-999b-4808-93f3-6136eb27ca6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:05.330212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89d949-999b-4808-93f3-6136eb27ca6d service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:05.330497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89d949-999b-4808-93f3-6136eb27ca6d service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:05.330773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89d949-999b-4808-93f3-6136eb27ca6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:05.341574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89d949-999b-4808-93f3-6136eb27ca6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:05.343873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89d949-999b-4808-93f3-6136eb27ca6d service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:05.344990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a67ac193a604f2aaf2500e542116347', 'request_id': 'req-4d603b25-ce97-4c06-9a3c-40369ad62bd4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.353829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a67ac193a604f2aaf2500e542116347', 'request_id': 'req-4d603b25-ce97-4c06-9a3c-40369ad62bd4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.361972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 975/1941] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:05.365310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89d949-999b-4808-93f3-6136eb27ca6d service placement] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:05.366861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a67ac193a604f2aaf2500e542116347', 'request_id': 'req-4d603b25-ce97-4c06-9a3c-40369ad62bd4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.370757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6c89d949-999b-4808-93f3-6136eb27ca6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:05.384474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] The token cache is being invalidate because project 3a67ac193a604f2aaf2500e542116347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:05.384789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.385527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.389810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89d949-999b-4808-93f3-6136eb27ca6d service placement] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:05.391489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 967/1942] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:05 2022] DELETE /identity/v3/projects/3a67ac193a604f2aaf2500e542116347 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:05.392823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:05.394589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:05.470016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:05.479991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:05.479991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:05.479991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:05.480355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] PATH_INFO: `/v3/users/dcd9e18442c24cce92cfa75abafad209` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:05.482285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcd9e18442c24cce92cfa75abafad209)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:05.498959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:05.500291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d603b25-ce97-4c06-9a3c-40369ad62bd4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:05.507938 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:05.508595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:05.508958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:05.509242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] PATH_INFO: `/v3/users/836f881962c74f11a24a5b39e091e021` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:05.510504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=836f881962c74f11a24a5b39e091e021)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:05.512746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:05.617987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836f881962c74f11a24a5b39e091e021'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.621241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcd9e18442c24cce92cfa75abafad209'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.622508 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcd9e18442c24cce92cfa75abafad209'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.627671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836f881962c74f11a24a5b39e091e021'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.628482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836f881962c74f11a24a5b39e091e021'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.632014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcd9e18442c24cce92cfa75abafad209'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.633148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcd9e18442c24cce92cfa75abafad209'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.637399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836f881962c74f11a24a5b39e091e021'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.638200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836f881962c74f11a24a5b39e091e021'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.644557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcd9e18442c24cce92cfa75abafad209'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:05.666082 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 968/1943] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:05 2022] DELETE /identity/v3/users/836f881962c74f11a24a5b39e091e021 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:05.713667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 976/1944] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:05 2022] DELETE /identity/v3/users/dcd9e18442c24cce92cfa75abafad209 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:05.805686 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:05.807241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:05.894519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2c948-121c-4b47-a9ef-4b2ad3dc6bc8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:05.961425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:05.962059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:05.962380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:05.962656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:05.964995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:05.988051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:05.990594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:06.007583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:06.044856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:06.071107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.073247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.085503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 969/1945] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:06.109625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef7988-3f1e-41d0-8445-5d1ab5036d43 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:06.115284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:06.115905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.116147 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.116422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] PATH_INFO: `/v3/projects/8645ab107c9b4d8bb4015aa8529c6ea5` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.117636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8645ab107c9b4d8bb4015aa8529c6ea5)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.119579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.169863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:06.172575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:06.184058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8645ab107c9b4d8bb4015aa8529c6ea5', 'request_id': 'req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.194588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8645ab107c9b4d8bb4015aa8529c6ea5', 'request_id': 'req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.197428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63622d-4d1d-4051-b3c1-1809ce2199ae service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:06.210309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8645ab107c9b4d8bb4015aa8529c6ea5', 'request_id': 'req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.233184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] The token cache is being invalidate because project 8645ab107c9b4d8bb4015aa8529c6ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:06.233808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.234626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a4fe5dd-6b08-4fe4-bdb4-1e630ecc8347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.236316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 977/1946] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:05 2022] DELETE /identity/v3/projects/8645ab107c9b4d8bb4015aa8529c6ea5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:06.242395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa5630-c29f-40c4-be03-520c872e54c7 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.242874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa5630-c29f-40c4-be03-520c872e54c7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.243158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa5630-c29f-40c4-be03-520c872e54c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.266415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-1cfa5630-c29f-40c4-be03-520c872e54c7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:06.267641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-1cfa5630-c29f-40c4-be03-520c872e54c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:06.299944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:06.300697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.301695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.302045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] PATH_INFO: `/v3/projects/cbe706371bed4efc8b0ea523bc9f4523` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.303705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe706371bed4efc8b0ea523bc9f4523)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.306108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.344041 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 978/1947] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:06.352862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68896764-8d94-48a2-b93d-5de0829983c6 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.353160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68896764-8d94-48a2-b93d-5de0829983c6 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.353427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68896764-8d94-48a2-b93d-5de0829983c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.388531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe706371bed4efc8b0ea523bc9f4523', 'request_id': 'req-82478d5f-1635-453a-86fd-45153bf6f136'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.398766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-68896764-8d94-48a2-b93d-5de0829983c6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:06.400085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-68896764-8d94-48a2-b93d-5de0829983c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:06.406922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe706371bed4efc8b0ea523bc9f4523', 'request_id': 'req-82478d5f-1635-453a-86fd-45153bf6f136'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.437048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe706371bed4efc8b0ea523bc9f4523', 'request_id': 'req-82478d5f-1635-453a-86fd-45153bf6f136'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.454454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] The token cache is being invalidate because project cbe706371bed4efc8b0ea523bc9f4523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:06.454758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.456136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.461470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 970/1948] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:06 2022] DELETE /identity/v3/projects/cbe706371bed4efc8b0ea523bc9f4523 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:06.465137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:06.468445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:06.472022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:06.490701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 979/1949] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:06.494447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68896764-8d94-48a2-b93d-5de0829983c6 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:06.505090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68896764-8d94-48a2-b93d-5de0829983c6 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:06.513780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b66562-e199-4ea9-ad93-b68647c4c992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:06.514752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b66562-e199-4ea9-ad93-b68647c4c992 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.515210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b66562-e199-4ea9-ad93-b68647c4c992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.515729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b66562-e199-4ea9-ad93-b68647c4c992 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.516748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b66562-e199-4ea9-ad93-b68647c4c992 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:06.517288 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b66562-e199-4ea9-ad93-b68647c4c992 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.519443 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b66562-e199-4ea9-ad93-b68647c4c992 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.537307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 980/1950] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:58:06.541822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b66562-e199-4ea9-ad93-b68647c4c992 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:06.553708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b66562-e199-4ea9-ad93-b68647c4c992 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:06.563614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be970b4-036f-41a8-875c-eabdb455f67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:06.564135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:06.564240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be970b4-036f-41a8-875c-eabdb455f67c admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.564499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be970b4-036f-41a8-875c-eabdb455f67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.565240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be970b4-036f-41a8-875c-eabdb455f67c admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.565896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be970b4-036f-41a8-875c-eabdb455f67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.568768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be970b4-036f-41a8-875c-eabdb455f67c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.571690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82478d5f-1635-453a-86fd-45153bf6f136 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:06.580178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:06.580946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.581396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.581749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] PATH_INFO: `/v3/users/b8537d7a48224120b7abc5ec2c587680` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.583594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8537d7a48224120b7abc5ec2c587680)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.585773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.608241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 981/1951] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:06 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:06.614755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be970b4-036f-41a8-875c-eabdb455f67c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:06.652582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8537d7a48224120b7abc5ec2c587680'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.660548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8537d7a48224120b7abc5ec2c587680'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.661507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8537d7a48224120b7abc5ec2c587680'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.669897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8537d7a48224120b7abc5ec2c587680'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.670970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8537d7a48224120b7abc5ec2c587680'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:06.687418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be970b4-036f-41a8-875c-eabdb455f67c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:06.689718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 971/1952] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:06 2022] DELETE /identity/v3/users/b8537d7a48224120b7abc5ec2c587680 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:06.693355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd3999-7e21-40f0-8dde-3fa82784c261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:06.693967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd3999-7e21-40f0-8dde-3fa82784c261 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.694247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd3999-7e21-40f0-8dde-3fa82784c261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.694520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd3999-7e21-40f0-8dde-3fa82784c261 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.695147 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd3999-7e21-40f0-8dde-3fa82784c261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.697145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd3999-7e21-40f0-8dde-3fa82784c261 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.722920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 982/1953] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:06 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:06.728042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:06.730317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:06.805093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd307e1-5d04-46ae-b8c9-6187e1062ac7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:06.809810 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcea8b-24c9-4906-bc1f-ea30c4c7412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:06.810422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcea8b-24c9-4906-bc1f-ea30c4c7412a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.810701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcea8b-24c9-4906-bc1f-ea30c4c7412a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.810972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcea8b-24c9-4906-bc1f-ea30c4c7412a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.811603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcea8b-24c9-4906-bc1f-ea30c4c7412a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.813287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcea8b-24c9-4906-bc1f-ea30c4c7412a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.824630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 972/1954] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:06.828586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd3999-7e21-40f0-8dde-3fa82784c261 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:06.839869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd3999-7e21-40f0-8dde-3fa82784c261 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:06.845854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f764d-625d-4acb-b234-5d84973119c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:06.846477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f764d-625d-4acb-b234-5d84973119c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.846771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f764d-625d-4acb-b234-5d84973119c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.847028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f764d-625d-4acb-b234-5d84973119c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.848418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f764d-625d-4acb-b234-5d84973119c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.859717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f764d-625d-4acb-b234-5d84973119c6 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.875646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 983/1955] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:58:06.879970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcea8b-24c9-4906-bc1f-ea30c4c7412a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:06.891719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcea8b-24c9-4906-bc1f-ea30c4c7412a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:06.897259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f1d93-2d96-45c5-bb13-900d0ab31ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:06.898086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f1d93-2d96-45c5-bb13-900d0ab31ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.898463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f1d93-2d96-45c5-bb13-900d0ab31ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.898832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f1d93-2d96-45c5-bb13-900d0ab31ce0 admin admin] PATH_INFO: `/v3/projects/7c6c5c7edd764214be9edb75361ab658/users/78b70b4913b248878075b8b1d8d6cbce/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.915460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f1d93-2d96-45c5-bb13-900d0ab31ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6c5c7edd764214be9edb75361ab658, user_id=78b70b4913b248878075b8b1d8d6cbce, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:06.918195 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f1d93-2d96-45c5-bb13-900d0ab31ce0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:06.935004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 973/1956] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:06 2022] PUT /identity/v3/projects/7c6c5c7edd764214be9edb75361ab658/users/78b70b4913b248878075b8b1d8d6cbce/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:06.946732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22463fc4-42e2-4cb0-bd22-99a03543d8e2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:06.947072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22463fc4-42e2-4cb0-bd22-99a03543d8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:06.947574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22463fc4-42e2-4cb0-bd22-99a03543d8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:06.979826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-22463fc4-42e2-4cb0-bd22-99a03543d8e2 None None] MFA Rules not processed for user `78b70b4913b248878075b8b1d8d6cbce`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:06.981098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-22463fc4-42e2-4cb0-bd22-99a03543d8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:07.024402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f1d93-2d96-45c5-bb13-900d0ab31ce0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:07.026309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-977f1d93-2d96-45c5-bb13-900d0ab31ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:07.075131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 984/1957] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:07.085951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22463fc4-42e2-4cb0-bd22-99a03543d8e2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:07.087317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-22463fc4-42e2-4cb0-bd22-99a03543d8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:07.096949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:07.116735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f1d93-2d96-45c5-bb13-900d0ab31ce0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:07.120471 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:07.122025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:07.122798 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.123083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.123646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] PATH_INFO: `/v3/projects/f860ad33172c46719049754bd174818f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.126019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f860ad33172c46719049754bd174818f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:07.127984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:07.165544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22463fc4-42e2-4cb0-bd22-99a03543d8e2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:07.178900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f860ad33172c46719049754bd174818f', 'request_id': 'req-98a3112a-dd79-4f1b-adf5-a0c2cab79217'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:07.184862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f860ad33172c46719049754bd174818f', 'request_id': 'req-98a3112a-dd79-4f1b-adf5-a0c2cab79217'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:07.192517 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f860ad33172c46719049754bd174818f', 'request_id': 'req-98a3112a-dd79-4f1b-adf5-a0c2cab79217'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:07.204829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] The token cache is being invalidate because project f860ad33172c46719049754bd174818f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:07.205132 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:07.205831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-98a3112a-dd79-4f1b-adf5-a0c2cab79217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:07.207856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 974/1958] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:07 2022] DELETE /identity/v3/projects/f860ad33172c46719049754bd174818f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:07.212746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63dfc6-4b89-4a2a-bd76-22eab036e4ca None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.213061 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63dfc6-4b89-4a2a-bd76-22eab036e4ca None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.213337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63dfc6-4b89-4a2a-bd76-22eab036e4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.233446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1d88e-055d-48f6-a6a8-90f77efbf392 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:07.234061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1d88e-055d-48f6-a6a8-90f77efbf392 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.234255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dd63dfc6-4b89-4a2a-bd76-22eab036e4ca None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:07.234362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1d88e-055d-48f6-a6a8-90f77efbf392 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.234604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1d88e-055d-48f6-a6a8-90f77efbf392 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.235733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dd63dfc6-4b89-4a2a-bd76-22eab036e4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:07.236374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e5f1d88e-055d-48f6-a6a8-90f77efbf392 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:07.299199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1d88e-055d-48f6-a6a8-90f77efbf392 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:07.300992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1d88e-055d-48f6-a6a8-90f77efbf392 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:07.302012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 975/1959] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:07.309739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395d77e-1736-48e8-a6fd-fb910c365fbc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.310026 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395d77e-1736-48e8-a6fd-fb910c365fbc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.310297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395d77e-1736-48e8-a6fd-fb910c365fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.310992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 985/1960] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:07.330181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a395d77e-1736-48e8-a6fd-fb910c365fbc None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:07.331409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a395d77e-1736-48e8-a6fd-fb910c365fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:07.339946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 976/1961] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:07.344227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1d88e-055d-48f6-a6a8-90f77efbf392 service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:07.355650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1d88e-055d-48f6-a6a8-90f77efbf392 service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:07.360643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454254f3-2df3-42f0-ba92-84bdd2b6ee20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:07.361270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454254f3-2df3-42f0-ba92-84bdd2b6ee20 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.361550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454254f3-2df3-42f0-ba92-84bdd2b6ee20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.361819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454254f3-2df3-42f0-ba92-84bdd2b6ee20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.362457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454254f3-2df3-42f0-ba92-84bdd2b6ee20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:07.362758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454254f3-2df3-42f0-ba92-84bdd2b6ee20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:07.364599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454254f3-2df3-42f0-ba92-84bdd2b6ee20 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:07.378744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 986/1962] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:58:07.382723 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395d77e-1736-48e8-a6fd-fb910c365fbc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:07.392825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395d77e-1736-48e8-a6fd-fb910c365fbc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:07.398322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0940-ea11-4880-89c9-3ee71427630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:07.398961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0940-ea11-4880-89c9-3ee71427630d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.399243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0940-ea11-4880-89c9-3ee71427630d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.399518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0940-ea11-4880-89c9-3ee71427630d admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.400163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf0940-ea11-4880-89c9-3ee71427630d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:07.402125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf0940-ea11-4880-89c9-3ee71427630d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:07.446052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 977/1963] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:07 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:07.451308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454254f3-2df3-42f0-ba92-84bdd2b6ee20 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:07.466308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454254f3-2df3-42f0-ba92-84bdd2b6ee20 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:07.554970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:07.590004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033f9a4-bb30-4812-b0e3-b75d56986d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:07.591348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f9a4-bb30-4812-b0e3-b75d56986d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.591931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f9a4-bb30-4812-b0e3-b75d56986d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.592503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f9a4-bb30-4812-b0e3-b75d56986d18 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.594059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033f9a4-bb30-4812-b0e3-b75d56986d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:07.599129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033f9a4-bb30-4812-b0e3-b75d56986d18 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:07.659422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 987/1964] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:07 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:07.661514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0940-ea11-4880-89c9-3ee71427630d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:07.682894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0940-ea11-4880-89c9-3ee71427630d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:07.692429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14286299-c75f-4635-8ec0-ae15b7fcce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:07.694245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14286299-c75f-4635-8ec0-ae15b7fcce0d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.694804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14286299-c75f-4635-8ec0-ae15b7fcce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.695931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14286299-c75f-4635-8ec0-ae15b7fcce0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.697511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14286299-c75f-4635-8ec0-ae15b7fcce0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:07.701229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14286299-c75f-4635-8ec0-ae15b7fcce0d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:07.721111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 978/1965] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:07.725637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033f9a4-bb30-4812-b0e3-b75d56986d18 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:07.745602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033f9a4-bb30-4812-b0e3-b75d56986d18 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:07.752419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b0ffc-b2d9-495f-8e6b-4bcdf57ed054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:07.753144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b0ffc-b2d9-495f-8e6b-4bcdf57ed054 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.753456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b0ffc-b2d9-495f-8e6b-4bcdf57ed054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.753716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b0ffc-b2d9-495f-8e6b-4bcdf57ed054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.755410 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b0ffc-b2d9-495f-8e6b-4bcdf57ed054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:07.757128 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b0ffc-b2d9-495f-8e6b-4bcdf57ed054 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:07.769580 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 988/1966] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:07.774359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14286299-c75f-4635-8ec0-ae15b7fcce0d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:07.788071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14286299-c75f-4635-8ec0-ae15b7fcce0d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:07.793030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a189fe-3d06-4a1d-8a9e-f542948969b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:07.793646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a189fe-3d06-4a1d-8a9e-f542948969b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.793979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a189fe-3d06-4a1d-8a9e-f542948969b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.794253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a189fe-3d06-4a1d-8a9e-f542948969b3 admin admin] PATH_INFO: `/v3/projects/2e8f604cf2b54550a90f748e7e0be91d/users/906eba5968674ef49cbefdd000724026/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.810921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a189fe-3d06-4a1d-8a9e-f542948969b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8f604cf2b54550a90f748e7e0be91d, user_id=906eba5968674ef49cbefdd000724026, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:07.813880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a189fe-3d06-4a1d-8a9e-f542948969b3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:07.831726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 979/1967] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:07 2022] PUT /identity/v3/projects/2e8f604cf2b54550a90f748e7e0be91d/users/906eba5968674ef49cbefdd000724026/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:07.845062 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e397463-fb1c-4e33-af88-bbe36bc0491d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:07.845931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e397463-fb1c-4e33-af88-bbe36bc0491d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:07.845931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e397463-fb1c-4e33-af88-bbe36bc0491d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:07.875166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9e397463-fb1c-4e33-af88-bbe36bc0491d None None] MFA Rules not processed for user `906eba5968674ef49cbefdd000724026`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:07.876402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9e397463-fb1c-4e33-af88-bbe36bc0491d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:07.948358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 989/1968] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:07.959768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a189fe-3d06-4a1d-8a9e-f542948969b3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:07.961490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d4a189fe-3d06-4a1d-8a9e-f542948969b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:07.980211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:08.021849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a189fe-3d06-4a1d-8a9e-f542948969b3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:08.025512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9c4b9-ec9a-4ea9-bcc0-3e0f76ee0460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:08.026185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9c4b9-ec9a-4ea9-bcc0-3e0f76ee0460 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:08.026475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9c4b9-ec9a-4ea9-bcc0-3e0f76ee0460 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:08.026744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9c4b9-ec9a-4ea9-bcc0-3e0f76ee0460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:08.035343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9c4b9-ec9a-4ea9-bcc0-3e0f76ee0460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:08.036941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9c4b9-ec9a-4ea9-bcc0-3e0f76ee0460 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:08.045926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 980/1969] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:14.958383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e397463-fb1c-4e33-af88-bbe36bc0491d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:15.004072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9c4b9-ec9a-4ea9-bcc0-3e0f76ee0460 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:15.013035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9c4b9-ec9a-4ea9-bcc0-3e0f76ee0460 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:15.017459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd05e4b-5451-4bd8-bbb0-d46b92355f5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:15.018171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd05e4b-5451-4bd8-bbb0-d46b92355f5c service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:15.018521 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd05e4b-5451-4bd8-bbb0-d46b92355f5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:15.018870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd05e4b-5451-4bd8-bbb0-d46b92355f5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:15.020661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2bd05e4b-5451-4bd8-bbb0-d46b92355f5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:15.037313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e397463-fb1c-4e33-af88-bbe36bc0491d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:15.043362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:15.043866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:15.044147 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:15.044422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] PATH_INFO: `/v3/users/906eba5968674ef49cbefdd000724026` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:15.045661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] RBAC: Authorizing `identity:delete_user(user_id=906eba5968674ef49cbefdd000724026)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:15.048059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:15.093404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd05e4b-5451-4bd8-bbb0-d46b92355f5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:15.094865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd05e4b-5451-4bd8-bbb0-d46b92355f5c service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:15.107081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906eba5968674ef49cbefdd000724026'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.107440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 981/1970] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:15.107762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906eba5968674ef49cbefdd000724026'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.115926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906eba5968674ef49cbefdd000724026'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.116511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906eba5968674ef49cbefdd000724026'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.124514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:15.126463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906eba5968674ef49cbefdd000724026'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.174223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 990/1971] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:14 2022] DELETE /identity/v3/users/906eba5968674ef49cbefdd000724026 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:15.175837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e3a2c-3b10-4612-9d3a-d2c749940159 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:15.176221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e3a2c-3b10-4612-9d3a-d2c749940159 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:15.177382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e3a2c-3b10-4612-9d3a-d2c749940159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:15.207051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-fd2e3a2c-3b10-4612-9d3a-d2c749940159 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:15.209216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fd2e3a2c-3b10-4612-9d3a-d2c749940159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:15.328114 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 982/1972] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:15.339455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:15.341360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:15.421096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee95db3-b66e-4718-951f-5e5d57d4b32e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:15.427244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f2d5b-79fc-434c-b084-0e2e3925fdaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:15.428164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f2d5b-79fc-434c-b084-0e2e3925fdaf service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:15.428469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f2d5b-79fc-434c-b084-0e2e3925fdaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:15.428789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f2d5b-79fc-434c-b084-0e2e3925fdaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:15.449961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f2d5b-79fc-434c-b084-0e2e3925fdaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:15.451828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f2d5b-79fc-434c-b084-0e2e3925fdaf service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:15.465137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 991/1973] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:15.803512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e3a2c-3b10-4612-9d3a-d2c749940159 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:15.804857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fd2e3a2c-3b10-4612-9d3a-d2c749940159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:15.821420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e3a2c-3b10-4612-9d3a-d2c749940159 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:15.826838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:15.827202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:15.827554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:15.827907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] PATH_INFO: `/v3/projects/2e8f604cf2b54550a90f748e7e0be91d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:15.829153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8f604cf2b54550a90f748e7e0be91d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:15.831472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:15.887184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8f604cf2b54550a90f748e7e0be91d', 'request_id': 'req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.889257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:15.894281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8f604cf2b54550a90f748e7e0be91d', 'request_id': 'req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.903460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8f604cf2b54550a90f748e7e0be91d', 'request_id': 'req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.915895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] The token cache is being invalidate because project 2e8f604cf2b54550a90f748e7e0be91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:15.916181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.916843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e3f09162-bf5d-4d2c-b6f8-b2dee2388665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:15.918429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 983/1974] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:15 2022] DELETE /identity/v3/projects/2e8f604cf2b54550a90f748e7e0be91d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:15.923595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf8da2-d3cc-4cbb-b5a9-02bd98ff72e7 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:15.924055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf8da2-d3cc-4cbb-b5a9-02bd98ff72e7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:15.924328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf8da2-d3cc-4cbb-b5a9-02bd98ff72e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:15.963299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-74bf8da2-d3cc-4cbb-b5a9-02bd98ff72e7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:15.964382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-74bf8da2-d3cc-4cbb-b5a9-02bd98ff72e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:16.076900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 992/1975] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:16.085336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c8a3a-500e-4233-b39a-4aefee2f4012 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.085739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c8a3a-500e-4233-b39a-4aefee2f4012 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.085952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c8a3a-500e-4233-b39a-4aefee2f4012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.116129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b32c8a3a-500e-4233-b39a-4aefee2f4012 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:16.117472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b32c8a3a-500e-4233-b39a-4aefee2f4012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:16.148025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 984/1976] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:16.152656 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf8da2-d3cc-4cbb-b5a9-02bd98ff72e7 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:16.181749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf8da2-d3cc-4cbb-b5a9-02bd98ff72e7 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:16.188537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bb1eb-c8c7-4b87-8415-c328e318675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:16.189207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bb1eb-c8c7-4b87-8415-c328e318675f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.189544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bb1eb-c8c7-4b87-8415-c328e318675f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.189824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bb1eb-c8c7-4b87-8415-c328e318675f admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.190891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bb1eb-c8c7-4b87-8415-c328e318675f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:16.191199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bb1eb-c8c7-4b87-8415-c328e318675f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:16.193688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bb1eb-c8c7-4b87-8415-c328e318675f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:16.206588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 993/1977] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:58:16.211094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c8a3a-500e-4233-b39a-4aefee2f4012 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:16.220423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c8a3a-500e-4233-b39a-4aefee2f4012 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:16.226511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd97c2-dc43-4a52-9291-14aeb8b53152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:16.227384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd97c2-dc43-4a52-9291-14aeb8b53152 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.227659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd97c2-dc43-4a52-9291-14aeb8b53152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.228070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd97c2-dc43-4a52-9291-14aeb8b53152 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.229218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd97c2-dc43-4a52-9291-14aeb8b53152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:16.231793 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd97c2-dc43-4a52-9291-14aeb8b53152 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:16.268428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 985/1978] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:16 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:16.269826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bb1eb-c8c7-4b87-8415-c328e318675f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:16.280098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bb1eb-c8c7-4b87-8415-c328e318675f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:16.337507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a8b3-8a06-4394-87e0-ee538c5f0cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:16.338197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a8b3-8a06-4394-87e0-ee538c5f0cab admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.338479 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a8b3-8a06-4394-87e0-ee538c5f0cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.338751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a8b3-8a06-4394-87e0-ee538c5f0cab admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.339443 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19a8b3-8a06-4394-87e0-ee538c5f0cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:16.341407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19a8b3-8a06-4394-87e0-ee538c5f0cab admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:16.357703 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:16.365092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 994/1979] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:16 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:16.369961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd97c2-dc43-4a52-9291-14aeb8b53152 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:16.381479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd97c2-dc43-4a52-9291-14aeb8b53152 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:16.386994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6a9cf-3414-4b7e-aa9b-61ce87e8fc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:16.387660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6a9cf-3414-4b7e-aa9b-61ce87e8fc25 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.387943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6a9cf-3414-4b7e-aa9b-61ce87e8fc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.388219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6a9cf-3414-4b7e-aa9b-61ce87e8fc25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.388907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6a9cf-3414-4b7e-aa9b-61ce87e8fc25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:16.390570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6a9cf-3414-4b7e-aa9b-61ce87e8fc25 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:16.401687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 986/1980] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:16.405677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a8b3-8a06-4394-87e0-ee538c5f0cab admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:16.415664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a8b3-8a06-4394-87e0-ee538c5f0cab admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:16.420209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5077e5c-558b-4cd9-8e5d-fccd6e324396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:16.420826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5077e5c-558b-4cd9-8e5d-fccd6e324396 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.421125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5077e5c-558b-4cd9-8e5d-fccd6e324396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.421399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5077e5c-558b-4cd9-8e5d-fccd6e324396 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.422030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5077e5c-558b-4cd9-8e5d-fccd6e324396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:16.423630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5077e5c-558b-4cd9-8e5d-fccd6e324396 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:16.434637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 995/1981] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:16.438663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6a9cf-3414-4b7e-aa9b-61ce87e8fc25 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:16.449940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6a9cf-3414-4b7e-aa9b-61ce87e8fc25 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:16.454553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949929c-fe3e-4537-9ab6-a2e9bece4d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:16.455199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949929c-fe3e-4537-9ab6-a2e9bece4d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.455482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949929c-fe3e-4537-9ab6-a2e9bece4d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.455754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949929c-fe3e-4537-9ab6-a2e9bece4d1b admin admin] PATH_INFO: `/v3/projects/179b9f65028648e18eeb74963c50122a/users/f4ecdf8ab7d84d35adb866f94aaf3eff/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.470040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949929c-fe3e-4537-9ab6-a2e9bece4d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=179b9f65028648e18eeb74963c50122a, user_id=f4ecdf8ab7d84d35adb866f94aaf3eff, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:16.472756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949929c-fe3e-4537-9ab6-a2e9bece4d1b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:16.487048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 987/1982] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:16 2022] PUT /identity/v3/projects/179b9f65028648e18eeb74963c50122a/users/f4ecdf8ab7d84d35adb866f94aaf3eff/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:16.492657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4ec3-96a1-4b12-9f00-6c7ecb1a546b None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.492980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4ec3-96a1-4b12-9f00-6c7ecb1a546b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.493260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4ec3-96a1-4b12-9f00-6c7ecb1a546b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.511878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7e8d4ec3-96a1-4b12-9f00-6c7ecb1a546b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:16.513111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7e8d4ec3-96a1-4b12-9f00-6c7ecb1a546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:16.572464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 996/1983] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:16.582378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949929c-fe3e-4537-9ab6-a2e9bece4d1b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:16.584264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a949929c-fe3e-4537-9ab6-a2e9bece4d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:16.648904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949929c-fe3e-4537-9ab6-a2e9bece4d1b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:16.650292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2f382-f715-45b2-b2f1-a8bc58023a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:16.650766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2f382-f715-45b2-b2f1-a8bc58023a12 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.651207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2f382-f715-45b2-b2f1-a8bc58023a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.651481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2f382-f715-45b2-b2f1-a8bc58023a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.661588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2f382-f715-45b2-b2f1-a8bc58023a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:16.663043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2f382-f715-45b2-b2f1-a8bc58023a12 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:16.664053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3f41c-b108-4b91-a843-bc9569a8c791 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:16.664719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3f41c-b108-4b91-a843-bc9569a8c791 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:16.664719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3f41c-b108-4b91-a843-bc9569a8c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:16.672730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 988/1984] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:16.685573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d5c3f41c-b108-4b91-a843-bc9569a8c791 None None] MFA Rules not processed for user `c1c8a563a78546fe9714d69913418c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:16.686802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d5c3f41c-b108-4b91-a843-bc9569a8c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:16.746435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 997/1985] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:17.566595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c4616-b64b-40a1-b0c3-f26ef56031b4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:17.568566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c4616-b64b-40a1-b0c3-f26ef56031b4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:17.568912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c4616-b64b-40a1-b0c3-f26ef56031b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:17.634522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c14c4616-b64b-40a1-b0c3-f26ef56031b4 None None] MFA Rules not processed for user `f4ecdf8ab7d84d35adb866f94aaf3eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:17.635762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c14c4616-b64b-40a1-b0c3-f26ef56031b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:17.654377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3f41c-b108-4b91-a843-bc9569a8c791 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:17.655958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d5c3f41c-b108-4b91-a843-bc9569a8c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:17.691205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3f41c-b108-4b91-a843-bc9569a8c791 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:17.704893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:17.710929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:17.711701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:17.712135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] PATH_INFO: `/v3/users/03251a7a44fa4f14ab3b5a2a8c3b6014` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:17.716027 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] RBAC: Authorizing `identity:delete_user(user_id=03251a7a44fa4f14ab3b5a2a8c3b6014)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:17.718828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:17.762543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 989/1986] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:17.762543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c4616-b64b-40a1-b0c3-f26ef56031b4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:17.763447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c14c4616-b64b-40a1-b0c3-f26ef56031b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:17.791671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c4616-b64b-40a1-b0c3-f26ef56031b4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:17.799404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:17.800435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:17.800724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:17.801082 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] PATH_INFO: `/v3/users/78b70b4913b248878075b8b1d8d6cbce` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:17.803574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03251a7a44fa4f14ab3b5a2a8c3b6014'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.805300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b70b4913b248878075b8b1d8d6cbce)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:17.806138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03251a7a44fa4f14ab3b5a2a8c3b6014'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.807959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:17.811278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:17.821832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03251a7a44fa4f14ab3b5a2a8c3b6014'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.824140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03251a7a44fa4f14ab3b5a2a8c3b6014'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.834272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03251a7a44fa4f14ab3b5a2a8c3b6014'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.863628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 998/1987] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:17 2022] DELETE /identity/v3/users/03251a7a44fa4f14ab3b5a2a8c3b6014 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:17.865115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:17.866675 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:17.877803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:17.878139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b70b4913b248878075b8b1d8d6cbce'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.893497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b70b4913b248878075b8b1d8d6cbce'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.894262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b70b4913b248878075b8b1d8d6cbce'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.904764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b70b4913b248878075b8b1d8d6cbce'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.905603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-931f9a1f-0bf1-46b2-98b6-13213d05ff0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b70b4913b248878075b8b1d8d6cbce'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:17.926852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 990/1988] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:17 2022] DELETE /identity/v3/users/78b70b4913b248878075b8b1d8d6cbce => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:17.928471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e01989-e5c4-47b9-90c6-c08b0a5cbea7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:17.928757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e01989-e5c4-47b9-90c6-c08b0a5cbea7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:17.929070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e01989-e5c4-47b9-90c6-c08b0a5cbea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:17.945963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e85e96-f353-4810-b1e8-405bc7fa298d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:17.955695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-42e01989-e5c4-47b9-90c6-c08b0a5cbea7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:17.956942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-42e01989-e5c4-47b9-90c6-c08b0a5cbea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.009215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:18.010402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:18.010836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:18.011496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:18.013830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.031230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 991/1989] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:18.031230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3255cc-ba31-464b-b51e-bbacd320ed27 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:18.031230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3255cc-ba31-464b-b51e-bbacd320ed27 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:18.031230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3255cc-ba31-464b-b51e-bbacd320ed27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:18.052001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9e3255cc-ba31-464b-b51e-bbacd320ed27 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:18.053491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9e3255cc-ba31-464b-b51e-bbacd320ed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.061980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 992/1990] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:18.065637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3255cc-ba31-464b-b51e-bbacd320ed27 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:18.067956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9e3255cc-ba31-464b-b51e-bbacd320ed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.100729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:18.102298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:18.116059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 999/1991] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:18.117033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:18.119339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.147858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3255cc-ba31-464b-b51e-bbacd320ed27 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:18.151779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f5101-e4dc-482f-85fd-5595fe189986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:18.152765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f5101-e4dc-482f-85fd-5595fe189986 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:18.154719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f5101-e4dc-482f-85fd-5595fe189986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:18.155000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f5101-e4dc-482f-85fd-5595fe189986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:18.168218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f5101-e4dc-482f-85fd-5595fe189986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:18.170078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f5101-e4dc-482f-85fd-5595fe189986 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:18.180986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 993/1992] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:18.194981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df2fb5-a493-4b3f-93ac-51fd7f269eba service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:18.206037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbce8d-2e5c-4e69-aca4-50d49df1e52b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:18.206926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbce8d-2e5c-4e69-aca4-50d49df1e52b service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:18.207314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbce8d-2e5c-4e69-aca4-50d49df1e52b service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:18.207692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbce8d-2e5c-4e69-aca4-50d49df1e52b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:18.222157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbce8d-2e5c-4e69-aca4-50d49df1e52b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:18.223929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbce8d-2e5c-4e69-aca4-50d49df1e52b service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:18.238456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1000/1993] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:18.614037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f5101-e4dc-482f-85fd-5595fe189986 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:18.616419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-759f5101-e4dc-482f-85fd-5595fe189986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.637216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f5101-e4dc-482f-85fd-5595fe189986 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:18.645745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:18.646781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:18.647531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:18.648027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] PATH_INFO: `/v3/projects/5999c1eac5434e18b1bd6430663a41de` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:18.649470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5999c1eac5434e18b1bd6430663a41de)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:18.653063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:18.670602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:18.715340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbce8d-2e5c-4e69-aca4-50d49df1e52b service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:18.722962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d0fbce8d-2e5c-4e69-aca4-50d49df1e52b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.748021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbce8d-2e5c-4e69-aca4-50d49df1e52b service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:18.751783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5999c1eac5434e18b1bd6430663a41de', 'request_id': 'req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.765780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:18.770001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5999c1eac5434e18b1bd6430663a41de', 'request_id': 'req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.781489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5999c1eac5434e18b1bd6430663a41de', 'request_id': 'req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.794257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] The token cache is being invalidate because project 5999c1eac5434e18b1bd6430663a41de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:18.794642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.795365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.797244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 994/1994] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:18 2022] DELETE /identity/v3/projects/5999c1eac5434e18b1bd6430663a41de => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:18.800835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:18.803359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.824451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:18.825035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:18.825896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:18.826335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] PATH_INFO: `/v3/projects/7c6c5c7edd764214be9edb75361ab658` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:18.827894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c6c5c7edd764214be9edb75361ab658)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:18.830361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:18.877807 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6618aa4-b84c-4d5a-9b5a-8c62d2e02535 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:18.888685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c6c5c7edd764214be9edb75361ab658', 'request_id': 'req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.894295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c6c5c7edd764214be9edb75361ab658', 'request_id': 'req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.902714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c6c5c7edd764214be9edb75361ab658', 'request_id': 'req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.914557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] The token cache is being invalidate because project 7c6c5c7edd764214be9edb75361ab658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:18.914850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.915504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-94f7b8e1-af08-44ee-9861-2ec23c2c4d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.917064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1001/1995] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:18 2022] DELETE /identity/v3/projects/7c6c5c7edd764214be9edb75361ab658 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:18.921814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2018f14-359b-4bfb-a96c-850f098eff6a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:18.922113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2018f14-359b-4bfb-a96c-850f098eff6a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:18.922414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2018f14-359b-4bfb-a96c-850f098eff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:18.935837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:18.936479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:18.936760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:18.937065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] PATH_INFO: `/v3/users/2372e419b08742e88dd3e79d280c76c3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:18.938271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2372e419b08742e88dd3e79d280c76c3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:18.940196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:18.941772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-c2018f14-359b-4bfb-a96c-850f098eff6a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:18.943692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c2018f14-359b-4bfb-a96c-850f098eff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.952113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1002/1996] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:18.960355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2775f-9b44-4cfd-acbe-e567a81f1487 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:18.960650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2775f-9b44-4cfd-acbe-e567a81f1487 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:18.960969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2775f-9b44-4cfd-acbe-e567a81f1487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:18.981220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-43e2775f-9b44-4cfd-acbe-e567a81f1487 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:18.982349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-43e2775f-9b44-4cfd-acbe-e567a81f1487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:18.990525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1003/1997] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:18.994803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2775f-9b44-4cfd-acbe-e567a81f1487 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:18.997089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2372e419b08742e88dd3e79d280c76c3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:18.999941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:19.005484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2775f-9b44-4cfd-acbe-e567a81f1487 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.006588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2372e419b08742e88dd3e79d280c76c3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.007439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2372e419b08742e88dd3e79d280c76c3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.010977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.011582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.011865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.012146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.012795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:19.013178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.014785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:19.016515 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2372e419b08742e88dd3e79d280c76c3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.017212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2372e419b08742e88dd3e79d280c76c3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.030884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1004/1998] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:58:19.032113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:19.034351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:19.038412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 995/1999] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:18 2022] DELETE /identity/v3/users/2372e419b08742e88dd3e79d280c76c3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:19.053580 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56416b50-b620-46b7-9bd4-b444e4c90536 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.143457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e136-7d14-43e5-a301-57c8db6e488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.144484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e136-7d14-43e5-a301-57c8db6e488f admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.144876 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e136-7d14-43e5-a301-57c8db6e488f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.145307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e136-7d14-43e5-a301-57c8db6e488f admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.146125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7e136-7d14-43e5-a301-57c8db6e488f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.148312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7e136-7d14-43e5-a301-57c8db6e488f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:19.186640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1005/2000] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:19 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:19.187795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:19.199427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b77d78-df61-40ef-94df-dd47c75b7ef4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.278969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955b49b-c8ce-4f17-9ddd-88137b7f19a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.279725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955b49b-c8ce-4f17-9ddd-88137b7f19a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.280088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955b49b-c8ce-4f17-9ddd-88137b7f19a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.280474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955b49b-c8ce-4f17-9ddd-88137b7f19a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.281400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955b49b-c8ce-4f17-9ddd-88137b7f19a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.283957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955b49b-c8ce-4f17-9ddd-88137b7f19a5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:19.315511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 996/2001] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:19 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:19.319846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e136-7d14-43e5-a301-57c8db6e488f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:19.338887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e136-7d14-43e5-a301-57c8db6e488f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.344672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bde60-6f41-4207-a05d-fbabe6860836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.345342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bde60-6f41-4207-a05d-fbabe6860836 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.345625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bde60-6f41-4207-a05d-fbabe6860836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.345898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bde60-6f41-4207-a05d-fbabe6860836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.346549 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702bde60-6f41-4207-a05d-fbabe6860836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.348429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702bde60-6f41-4207-a05d-fbabe6860836 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:19.350512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:19.365830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1006/2002] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:19.369368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955b49b-c8ce-4f17-9ddd-88137b7f19a5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:19.381087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955b49b-c8ce-4f17-9ddd-88137b7f19a5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.387298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529e354-fd87-4674-b88f-d31b15dc98da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.387918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529e354-fd87-4674-b88f-d31b15dc98da admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.388199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529e354-fd87-4674-b88f-d31b15dc98da admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.388613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529e354-fd87-4674-b88f-d31b15dc98da admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.389571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529e354-fd87-4674-b88f-d31b15dc98da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.391850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529e354-fd87-4674-b88f-d31b15dc98da admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:19.403779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 997/2003] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:19.408300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bde60-6f41-4207-a05d-fbabe6860836 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:19.414650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529e354-fd87-4674-b88f-d31b15dc98da admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:19.416379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5529e354-fd87-4674-b88f-d31b15dc98da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:19.425574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bde60-6f41-4207-a05d-fbabe6860836 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.432331 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529e354-fd87-4674-b88f-d31b15dc98da admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.437944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.438428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.438773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.439201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] PATH_INFO: `/v3/projects/39d5acde040b4c578930417d5e5f0841` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.440548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d5acde040b4c578930417d5e5f0841)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.442766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:19.454045 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007ef74-7daf-47fb-a3c5-121355b49d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.454818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007ef74-7daf-47fb-a3c5-121355b49d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.455225 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007ef74-7daf-47fb-a3c5-121355b49d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.455505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007ef74-7daf-47fb-a3c5-121355b49d31 admin admin] PATH_INFO: `/v3/projects/af5edc07bdee4a58b7faa9543639b301/users/c6832743c95843278fb3ad0ea2965327/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.475140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007ef74-7daf-47fb-a3c5-121355b49d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af5edc07bdee4a58b7faa9543639b301, user_id=c6832743c95843278fb3ad0ea2965327, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.477978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007ef74-7daf-47fb-a3c5-121355b49d31 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:19.499140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1007/2004] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:19 2022] PUT /identity/v3/projects/af5edc07bdee4a58b7faa9543639b301/users/c6832743c95843278fb3ad0ea2965327/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:19.513085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef019f-9945-4adc-b974-8432e5114053 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.513403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef019f-9945-4adc-b974-8432e5114053 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.513805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef019f-9945-4adc-b974-8432e5114053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.520041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d5acde040b4c578930417d5e5f0841', 'request_id': 'req-8ebed386-da02-4f7d-9c22-548b20299d56'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.529611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d5acde040b4c578930417d5e5f0841', 'request_id': 'req-8ebed386-da02-4f7d-9c22-548b20299d56'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.541177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d5acde040b4c578930417d5e5f0841', 'request_id': 'req-8ebed386-da02-4f7d-9c22-548b20299d56'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.561784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-67ef019f-9945-4adc-b974-8432e5114053 None None] MFA Rules not processed for user `c6832743c95843278fb3ad0ea2965327`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:19.563080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-67ef019f-9945-4adc-b974-8432e5114053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:19.574007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] The token cache is being invalidate because project 39d5acde040b4c578930417d5e5f0841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:19.574385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.575128 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8ebed386-da02-4f7d-9c22-548b20299d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.577134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 998/2005] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:19 2022] DELETE /identity/v3/projects/39d5acde040b4c578930417d5e5f0841 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:19.578495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690c3c2-0410-487b-9bd9-de984d212257 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.578876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690c3c2-0410-487b-9bd9-de984d212257 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.579421 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690c3c2-0410-487b-9bd9-de984d212257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.600407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5690c3c2-0410-487b-9bd9-de984d212257 None None] MFA Rules not processed for user `f4ecdf8ab7d84d35adb866f94aaf3eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:19.601836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5690c3c2-0410-487b-9bd9-de984d212257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:19.628574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:19.646006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1008/2006] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:19.647400 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef019f-9945-4adc-b974-8432e5114053 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:19.648790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-67ef019f-9945-4adc-b974-8432e5114053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:19.672463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 999/2007] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:19.673588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690c3c2-0410-487b-9bd9-de984d212257 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:19.679799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5690c3c2-0410-487b-9bd9-de984d212257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:19.742334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef019f-9945-4adc-b974-8432e5114053 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.748675 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.749345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.749625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.749889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] PATH_INFO: `/v3/users/892d708a05fa429ba84213f9716c576a` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.751026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=892d708a05fa429ba84213f9716c576a)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.753696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:19.769609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690c3c2-0410-487b-9bd9-de984d212257 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.776015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fddc98-35c0-442a-81fc-3d8929ebf97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.776638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fddc98-35c0-442a-81fc-3d8929ebf97b admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.776950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fddc98-35c0-442a-81fc-3d8929ebf97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.777239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fddc98-35c0-442a-81fc-3d8929ebf97b admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.777864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fddc98-35c0-442a-81fc-3d8929ebf97b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.779776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fddc98-35c0-442a-81fc-3d8929ebf97b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:19.825872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1000/2008] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:19 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:19.827455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fddc98-35c0-442a-81fc-3d8929ebf97b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:19.840955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892d708a05fa429ba84213f9716c576a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.842636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892d708a05fa429ba84213f9716c576a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.851835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892d708a05fa429ba84213f9716c576a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.852880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892d708a05fa429ba84213f9716c576a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.862075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892d708a05fa429ba84213f9716c576a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:19.863476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:19.894284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1009/2009] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:19 2022] DELETE /identity/v3/users/892d708a05fa429ba84213f9716c576a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:19.910083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fddc98-35c0-442a-81fc-3d8929ebf97b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:19.986787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0adc4-8b90-4a5c-8246-e7d741946e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:19.987504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0adc4-8b90-4a5c-8246-e7d741946e36 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:19.987786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0adc4-8b90-4a5c-8246-e7d741946e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:19.988117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0adc4-8b90-4a5c-8246-e7d741946e36 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:19.989650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0adc4-8b90-4a5c-8246-e7d741946e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:19.992823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0adc4-8b90-4a5c-8246-e7d741946e36 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:20.007969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:20.027755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1001/2010] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:19 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:20.043151 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:20.048557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:20.080264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef42405-9c3c-4620-91e5-af9a3a03fcbd admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:20.086633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f37cd-7499-4d6f-88be-70bbc0b4f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:20.087959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f37cd-7499-4d6f-88be-70bbc0b4f76f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.088315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f37cd-7499-4d6f-88be-70bbc0b4f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.088658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f37cd-7499-4d6f-88be-70bbc0b4f76f admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.089572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f37cd-7499-4d6f-88be-70bbc0b4f76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:20.091268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f37cd-7499-4d6f-88be-70bbc0b4f76f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:20.104851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1010/2011] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:20.107640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0adc4-8b90-4a5c-8246-e7d741946e36 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:20.120948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0adc4-8b90-4a5c-8246-e7d741946e36 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:20.127498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b567c-74df-4277-b852-a027acd337af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:20.128177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b567c-74df-4277-b852-a027acd337af admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.128453 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b567c-74df-4277-b852-a027acd337af admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.128726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b567c-74df-4277-b852-a027acd337af admin admin] PATH_INFO: `/v3/projects/2d8d04c44a994b6283d407737d942962/users/2eeca8f13001426eab204f28ca9c4e54/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.162533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b567c-74df-4277-b852-a027acd337af admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d8d04c44a994b6283d407737d942962, user_id=2eeca8f13001426eab204f28ca9c4e54, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:20.165784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b567c-74df-4277-b852-a027acd337af admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:20.189456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f37cd-7499-4d6f-88be-70bbc0b4f76f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:20.189932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1002/2012] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:20 2022] PUT /identity/v3/projects/2d8d04c44a994b6283d407737d942962/users/2eeca8f13001426eab204f28ca9c4e54/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:20.201831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f37cd-7499-4d6f-88be-70bbc0b4f76f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:20.283145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435d6e4-c63b-4521-8da8-c5da65a277d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:20.283765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435d6e4-c63b-4521-8da8-c5da65a277d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.284238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435d6e4-c63b-4521-8da8-c5da65a277d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.284388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435d6e4-c63b-4521-8da8-c5da65a277d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.285068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435d6e4-c63b-4521-8da8-c5da65a277d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:20.287524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435d6e4-c63b-4521-8da8-c5da65a277d2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:20.299739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1011/2013] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:20.304604 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b567c-74df-4277-b852-a027acd337af admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:20.316437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b567c-74df-4277-b852-a027acd337af admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:20.322673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584958f1-d1f1-4b07-81a1-09464d3d13de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:20.323610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584958f1-d1f1-4b07-81a1-09464d3d13de admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.323886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584958f1-d1f1-4b07-81a1-09464d3d13de admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.324162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584958f1-d1f1-4b07-81a1-09464d3d13de admin admin] PATH_INFO: `/v3/projects/2d8d04c44a994b6283d407737d942962/users/2eeca8f13001426eab204f28ca9c4e54/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.327708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584958f1-d1f1-4b07-81a1-09464d3d13de admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d8d04c44a994b6283d407737d942962, user_id=2eeca8f13001426eab204f28ca9c4e54, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:20.330897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584958f1-d1f1-4b07-81a1-09464d3d13de admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:20.356870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1003/2014] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:20 2022] PUT /identity/v3/projects/2d8d04c44a994b6283d407737d942962/users/2eeca8f13001426eab204f28ca9c4e54/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:20.360460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3b1e3-d27e-4cd8-8243-784e2528133a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.360866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3b1e3-d27e-4cd8-8243-784e2528133a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.361175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3b1e3-d27e-4cd8-8243-784e2528133a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.393508 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-79d3b1e3-d27e-4cd8-8243-784e2528133a None None] MFA Rules not processed for user `2eeca8f13001426eab204f28ca9c4e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:20.394790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-79d3b1e3-d27e-4cd8-8243-784e2528133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:20.409107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584958f1-d1f1-4b07-81a1-09464d3d13de admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:20.412125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-584958f1-d1f1-4b07-81a1-09464d3d13de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:20.518410 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1012/2015] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:20.523540 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584958f1-d1f1-4b07-81a1-09464d3d13de admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:20.527999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:20.528645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.528985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.529263 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] PATH_INFO: `/v3/projects/93c5056dda384cd1b38ea57641447e81` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.530390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c5056dda384cd1b38ea57641447e81)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:20.531750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3b1e3-d27e-4cd8-8243-784e2528133a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:20.532354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:20.535700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-79d3b1e3-d27e-4cd8-8243-784e2528133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:20.629388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3b1e3-d27e-4cd8-8243-784e2528133a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:20.634339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:20.639346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c5056dda384cd1b38ea57641447e81', 'request_id': 'req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:20.650311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c5056dda384cd1b38ea57641447e81', 'request_id': 'req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:20.666963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c5056dda384cd1b38ea57641447e81', 'request_id': 'req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:20.695647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] The token cache is being invalidate because project 93c5056dda384cd1b38ea57641447e81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:20.696218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:20.697115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f4adb0fe-24c6-48d2-b16c-e6c4f9e72b94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:20.704487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1004/2016] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:20 2022] DELETE /identity/v3/projects/93c5056dda384cd1b38ea57641447e81 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:20.706245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9fc04-74d0-454f-867a-18da6a2e014c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.706536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9fc04-74d0-454f-867a-18da6a2e014c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.706817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9fc04-74d0-454f-867a-18da6a2e014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.721753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ac83e-9703-4070-b809-847942670ca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:20.722382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ac83e-9703-4070-b809-847942670ca5 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.722664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ac83e-9703-4070-b809-847942670ca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.722935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ac83e-9703-4070-b809-847942670ca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.726525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f06ac83e-9703-4070-b809-847942670ca5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:20.734993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0da9fc04-74d0-454f-867a-18da6a2e014c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:20.736659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0da9fc04-74d0-454f-867a-18da6a2e014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:20.786900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:20.808913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1005/2017] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:20.816449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188792c-0ab6-465a-8a17-4b52fe44f1df None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.816737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188792c-0ab6-465a-8a17-4b52fe44f1df None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.817094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188792c-0ab6-465a-8a17-4b52fe44f1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.823895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ac83e-9703-4070-b809-847942670ca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:20.827189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ac83e-9703-4070-b809-847942670ca5 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:20.840302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e188792c-0ab6-465a-8a17-4b52fe44f1df None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:20.841350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1013/2018] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:20.843419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e188792c-0ab6-465a-8a17-4b52fe44f1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:20.852575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1006/2019] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:20.857777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ac83e-9703-4070-b809-847942670ca5 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:20.870826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ac83e-9703-4070-b809-847942670ca5 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:20.875707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dbf21-a1ed-4fd5-b151-6b2499503d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:20.876457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dbf21-a1ed-4fd5-b151-6b2499503d33 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.877890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dbf21-a1ed-4fd5-b151-6b2499503d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.878145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dbf21-a1ed-4fd5-b151-6b2499503d33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.878942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dbf21-a1ed-4fd5-b151-6b2499503d33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:20.879331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dbf21-a1ed-4fd5-b151-6b2499503d33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:20.882622 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dbf21-a1ed-4fd5-b151-6b2499503d33 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:20.900287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1014/2020] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:58:20.904446 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188792c-0ab6-465a-8a17-4b52fe44f1df None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:20.917476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188792c-0ab6-465a-8a17-4b52fe44f1df None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:20.924439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476aa3e-5c85-479b-a2bd-c67499bdb1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:20.925676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476aa3e-5c85-479b-a2bd-c67499bdb1ae admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:20.926156 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476aa3e-5c85-479b-a2bd-c67499bdb1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:20.927051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476aa3e-5c85-479b-a2bd-c67499bdb1ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:20.927783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476aa3e-5c85-479b-a2bd-c67499bdb1ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:20.931323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476aa3e-5c85-479b-a2bd-c67499bdb1ae admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.002768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1007/2021] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:20 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:21.007448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dbf21-a1ed-4fd5-b151-6b2499503d33 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.017644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dbf21-a1ed-4fd5-b151-6b2499503d33 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.146518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faf543-a915-4b0c-a223-e90efa4e5c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.147167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faf543-a915-4b0c-a223-e90efa4e5c9b admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.147985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faf543-a915-4b0c-a223-e90efa4e5c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.148506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faf543-a915-4b0c-a223-e90efa4e5c9b admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.149914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9faf543-a915-4b0c-a223-e90efa4e5c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.151984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9faf543-a915-4b0c-a223-e90efa4e5c9b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.185243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1015/2022] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:21 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:21.186528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476aa3e-5c85-479b-a2bd-c67499bdb1ae admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.201018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476aa3e-5c85-479b-a2bd-c67499bdb1ae admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.209605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef3444-d512-46bf-be98-4bbedb58968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.210493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef3444-d512-46bf-be98-4bbedb58968a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.211087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef3444-d512-46bf-be98-4bbedb58968a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.211713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef3444-d512-46bf-be98-4bbedb58968a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.212698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef3444-d512-46bf-be98-4bbedb58968a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.214638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef3444-d512-46bf-be98-4bbedb58968a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.232852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1008/2023] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:21.234572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faf543-a915-4b0c-a223-e90efa4e5c9b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.249814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faf543-a915-4b0c-a223-e90efa4e5c9b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.259011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51457943-2da1-4dfd-be61-47543ff44c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.259588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51457943-2da1-4dfd-be61-47543ff44c9f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.259915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51457943-2da1-4dfd-be61-47543ff44c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.260281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51457943-2da1-4dfd-be61-47543ff44c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.260967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51457943-2da1-4dfd-be61-47543ff44c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.262933 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51457943-2da1-4dfd-be61-47543ff44c9f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.284055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1016/2024] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:21.287424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef3444-d512-46bf-be98-4bbedb58968a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.302832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef3444-d512-46bf-be98-4bbedb58968a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.310200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c3aff-0ef8-47fb-9e66-829c69d42422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.311154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c3aff-0ef8-47fb-9e66-829c69d42422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.311154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c3aff-0ef8-47fb-9e66-829c69d42422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.311476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c3aff-0ef8-47fb-9e66-829c69d42422 admin admin] PATH_INFO: `/v3/projects/f1bcc1ab61f144f1a1622a71527a576c/users/bd23f3bf159d4d89965d89bb45031425/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.329980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c3aff-0ef8-47fb-9e66-829c69d42422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1bcc1ab61f144f1a1622a71527a576c, user_id=bd23f3bf159d4d89965d89bb45031425, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.333474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c3aff-0ef8-47fb-9e66-829c69d42422 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.383281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1009/2025] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:21 2022] PUT /identity/v3/projects/f1bcc1ab61f144f1a1622a71527a576c/users/bd23f3bf159d4d89965d89bb45031425/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:21.396400 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1087e-4be3-4ebb-bdb8-c41c6ead453b None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.396728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1087e-4be3-4ebb-bdb8-c41c6ead453b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.397100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1087e-4be3-4ebb-bdb8-c41c6ead453b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.414207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c3aff-0ef8-47fb-9e66-829c69d42422 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.416464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c58c3aff-0ef8-47fb-9e66-829c69d42422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:21.459017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d3e1087e-4be3-4ebb-bdb8-c41c6ead453b None None] MFA Rules not processed for user `bd23f3bf159d4d89965d89bb45031425`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:21.460837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d3e1087e-4be3-4ebb-bdb8-c41c6ead453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:21.476719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:21.545510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c3aff-0ef8-47fb-9e66-829c69d42422 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.563327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.564601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.564914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.565189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] PATH_INFO: `/v3/users/c6832743c95843278fb3ad0ea2965327` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.570055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6832743c95843278fb3ad0ea2965327)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.575981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.611517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1017/2026] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:21.616760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1087e-4be3-4ebb-bdb8-c41c6ead453b None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.635264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1087e-4be3-4ebb-bdb8-c41c6ead453b None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.645715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa7565-282c-41f8-890b-85ecce1ededc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.646680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa7565-282c-41f8-890b-85ecce1ededc admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.646927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa7565-282c-41f8-890b-85ecce1ededc admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.647105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa7565-282c-41f8-890b-85ecce1ededc admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.648097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa7565-282c-41f8-890b-85ecce1ededc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.650582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa7565-282c-41f8-890b-85ecce1ededc admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.672925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6832743c95843278fb3ad0ea2965327'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:21.686226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6832743c95843278fb3ad0ea2965327'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:21.686931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6832743c95843278fb3ad0ea2965327'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:21.687619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1018/2027] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:21 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:21.691853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa7565-282c-41f8-890b-85ecce1ededc admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.705268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6832743c95843278fb3ad0ea2965327'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:21.706629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-51095946-26cc-4de9-b1b5-0d832f5e27b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6832743c95843278fb3ad0ea2965327'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:21.730942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1010/2028] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:21 2022] DELETE /identity/v3/users/c6832743c95843278fb3ad0ea2965327 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:21.736404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e40539-01d7-425f-b73f-3d86478aff54 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.736691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e40539-01d7-425f-b73f-3d86478aff54 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.737011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e40539-01d7-425f-b73f-3d86478aff54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.762205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:21.783454 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa7565-282c-41f8-890b-85ecce1ededc admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.784721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-34e40539-01d7-425f-b73f-3d86478aff54 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:21.788353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-34e40539-01d7-425f-b73f-3d86478aff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:21.877592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e0dc2-0ed4-4d36-b7d0-475156f3a836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.878562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e0dc2-0ed4-4d36-b7d0-475156f3a836 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.878849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e0dc2-0ed4-4d36-b7d0-475156f3a836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.879121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e0dc2-0ed4-4d36-b7d0-475156f3a836 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.880071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e0dc2-0ed4-4d36-b7d0-475156f3a836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.881503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1011/2029] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:21.882677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e0dc2-0ed4-4d36-b7d0-475156f3a836 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.891852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e40539-01d7-425f-b73f-3d86478aff54 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.893400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-34e40539-01d7-425f-b73f-3d86478aff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:21.908660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1019/2030] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:21 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:21.913199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e0dc2-0ed4-4d36-b7d0-475156f3a836 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.915396 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-de8e0dc2-0ed4-4d36-b7d0-475156f3a836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:21.930823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e0dc2-0ed4-4d36-b7d0-475156f3a836 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.936331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1b921-2fd7-4d97-9519-b01e5cda9743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.936980 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1b921-2fd7-4d97-9519-b01e5cda9743 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.937666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1b921-2fd7-4d97-9519-b01e5cda9743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.937936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1b921-2fd7-4d97-9519-b01e5cda9743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.938602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1b921-2fd7-4d97-9519-b01e5cda9743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.940562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1b921-2fd7-4d97-9519-b01e5cda9743 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.952570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1020/2031] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:58:21.956432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1b921-2fd7-4d97-9519-b01e5cda9743 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:21.960949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e40539-01d7-425f-b73f-3d86478aff54 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.965852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d024a5d-44d2-4b28-999f-d0dcb040db44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.966483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d024a5d-44d2-4b28-999f-d0dcb040db44 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.966766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d024a5d-44d2-4b28-999f-d0dcb040db44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.967507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d024a5d-44d2-4b28-999f-d0dcb040db44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.968285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1b921-2fd7-4d97-9519-b01e5cda9743 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:21.973091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172878d3-fa4d-4273-8e38-2426a1a05353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:21.974103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172878d3-fa4d-4273-8e38-2426a1a05353 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:21.974493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172878d3-fa4d-4273-8e38-2426a1a05353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:21.974823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172878d3-fa4d-4273-8e38-2426a1a05353 admin admin] PATH_INFO: `/v3/projects/4b1d9e3cf9034b6ca2d016823fc4884f/users/bce8045e9a6f4e0d89130ba3bf0ca8b8/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:21.980277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d024a5d-44d2-4b28-999f-d0dcb040db44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.981790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d024a5d-44d2-4b28-999f-d0dcb040db44 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:21.988749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:21.992631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172878d3-fa4d-4273-8e38-2426a1a05353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b1d9e3cf9034b6ca2d016823fc4884f, user_id=bce8045e9a6f4e0d89130ba3bf0ca8b8, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:21.992789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1012/2032] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:21.995181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172878d3-fa4d-4273-8e38-2426a1a05353 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:22.011702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1021/2033] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:21 2022] PUT /identity/v3/projects/4b1d9e3cf9034b6ca2d016823fc4884f/users/bce8045e9a6f4e0d89130ba3bf0ca8b8/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:22.015697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d024a5d-44d2-4b28-999f-d0dcb040db44 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:22.026669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d024a5d-44d2-4b28-999f-d0dcb040db44 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:22.087097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639e47b-3b62-4512-94c1-1ff311cb2329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:22.087716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639e47b-3b62-4512-94c1-1ff311cb2329 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:22.087997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639e47b-3b62-4512-94c1-1ff311cb2329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:22.088272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639e47b-3b62-4512-94c1-1ff311cb2329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:22.089113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639e47b-3b62-4512-94c1-1ff311cb2329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:22.090891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639e47b-3b62-4512-94c1-1ff311cb2329 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:22.101846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1013/2034] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:22.105802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172878d3-fa4d-4273-8e38-2426a1a05353 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:22.116901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172878d3-fa4d-4273-8e38-2426a1a05353 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:22.121544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb1af9-8e19-4365-9b28-396282b72df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:22.122202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb1af9-8e19-4365-9b28-396282b72df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:22.122480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb1af9-8e19-4365-9b28-396282b72df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:22.122754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb1af9-8e19-4365-9b28-396282b72df2 admin admin] PATH_INFO: `/v3/projects/4b1d9e3cf9034b6ca2d016823fc4884f/users/bce8045e9a6f4e0d89130ba3bf0ca8b8/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:22.124648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb1af9-8e19-4365-9b28-396282b72df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b1d9e3cf9034b6ca2d016823fc4884f, user_id=bce8045e9a6f4e0d89130ba3bf0ca8b8, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:22.127331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb1af9-8e19-4365-9b28-396282b72df2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:22.143870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1022/2035] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:22 2022] PUT /identity/v3/projects/4b1d9e3cf9034b6ca2d016823fc4884f/users/bce8045e9a6f4e0d89130ba3bf0ca8b8/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:22.153918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e2ab-b0f8-4032-bede-b97afa8860cd None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:22.154213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e2ab-b0f8-4032-bede-b97afa8860cd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:22.154489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e2ab-b0f8-4032-bede-b97afa8860cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:22.183934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dfd3e2ab-b0f8-4032-bede-b97afa8860cd None None] MFA Rules not processed for user `bce8045e9a6f4e0d89130ba3bf0ca8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:22.185182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dfd3e2ab-b0f8-4032-bede-b97afa8860cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:22.223842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:22.276727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1014/2036] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:22.291953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb1af9-8e19-4365-9b28-396282b72df2 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:22.293737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-59eb1af9-8e19-4365-9b28-396282b72df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:22.381736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb1af9-8e19-4365-9b28-396282b72df2 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:22.385555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51704c93-14f4-4a51-8047-bec1e00ee395 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:22.387159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51704c93-14f4-4a51-8047-bec1e00ee395 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:22.387565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51704c93-14f4-4a51-8047-bec1e00ee395 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:22.387905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51704c93-14f4-4a51-8047-bec1e00ee395 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:22.405927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51704c93-14f4-4a51-8047-bec1e00ee395 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:22.405927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51704c93-14f4-4a51-8047-bec1e00ee395 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:22.423392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1023/2037] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:22.466979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3e2ab-b0f8-4032-bede-b97afa8860cd None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:22.469770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dfd3e2ab-b0f8-4032-bede-b97afa8860cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:22.581627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3e2ab-b0f8-4032-bede-b97afa8860cd None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:22.596354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:22.597115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:22.597335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:22.597611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] PATH_INFO: `/v3/projects/af5edc07bdee4a58b7faa9543639b301` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:22.600587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af5edc07bdee4a58b7faa9543639b301)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:22.602593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:22.698279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af5edc07bdee4a58b7faa9543639b301', 'request_id': 'req-fe34aa14-5292-4625-b67f-4fda7bd9aee1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.705799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af5edc07bdee4a58b7faa9543639b301', 'request_id': 'req-fe34aa14-5292-4625-b67f-4fda7bd9aee1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.718983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af5edc07bdee4a58b7faa9543639b301', 'request_id': 'req-fe34aa14-5292-4625-b67f-4fda7bd9aee1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.739667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] The token cache is being invalidate because project af5edc07bdee4a58b7faa9543639b301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:22.740123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.740826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.745442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1015/2038] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:22 2022] DELETE /identity/v3/projects/af5edc07bdee4a58b7faa9543639b301 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:22.747164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51704c93-14f4-4a51-8047-bec1e00ee395 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:22.762552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51704c93-14f4-4a51-8047-bec1e00ee395 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:22.852037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:22.853399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:22.853756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:22.854098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] PATH_INFO: `/v3/users/2eeca8f13001426eab204f28ca9c4e54` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:22.858719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eeca8f13001426eab204f28ca9c4e54)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:22.860718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:22.927032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eeca8f13001426eab204f28ca9c4e54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.927856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eeca8f13001426eab204f28ca9c4e54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.938791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eeca8f13001426eab204f28ca9c4e54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.938791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eeca8f13001426eab204f28ca9c4e54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.956861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eeca8f13001426eab204f28ca9c4e54'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:22.967071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:22.971979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1024/2039] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:22 2022] DELETE /identity/v3/users/2eeca8f13001426eab204f28ca9c4e54 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:23.150291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.151704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:23.164739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34aa14-5292-4625-b67f-4fda7bd9aee1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:23.225110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:23.225762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.226045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.226321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] PATH_INFO: `/v3/users/bd23f3bf159d4d89965d89bb45031425` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.227499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd23f3bf159d4d89965d89bb45031425)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:23.229516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:23.231224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:23.261997 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.263740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:23.286533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c2361-e894-4599-8d6a-bf426f3662fe admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:23.295840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:23.297158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.297903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.298654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] PATH_INFO: `/v3/projects/2d8d04c44a994b6283d407737d942962` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.300726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d8d04c44a994b6283d407737d942962)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:23.302488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd23f3bf159d4d89965d89bb45031425'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.304401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:23.311618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd23f3bf159d4d89965d89bb45031425'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.312571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd23f3bf159d4d89965d89bb45031425'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.324854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd23f3bf159d4d89965d89bb45031425'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.325874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e19e266-42b0-4f10-9dbd-8e35c98bff22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd23f3bf159d4d89965d89bb45031425'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.350970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1016/2040] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:23 2022] DELETE /identity/v3/users/bd23f3bf159d4d89965d89bb45031425 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:23.356038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f1af1-cff8-4602-9c5b-921777736c0f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.356452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f1af1-cff8-4602-9c5b-921777736c0f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.356845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f1af1-cff8-4602-9c5b-921777736c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.372504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d8d04c44a994b6283d407737d942962', 'request_id': 'req-8ed6940b-b4f7-4d88-8894-5320cd0425cc'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.378110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ca9f1af1-cff8-4602-9c5b-921777736c0f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:23.378492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d8d04c44a994b6283d407737d942962', 'request_id': 'req-8ed6940b-b4f7-4d88-8894-5320cd0425cc'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.379319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ca9f1af1-cff8-4602-9c5b-921777736c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:23.386190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d8d04c44a994b6283d407737d942962', 'request_id': 'req-8ed6940b-b4f7-4d88-8894-5320cd0425cc'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.398459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] The token cache is being invalidate because project 2d8d04c44a994b6283d407737d942962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:23.398744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.399498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8ed6940b-b4f7-4d88-8894-5320cd0425cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:23.401011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1025/2041] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:23 2022] DELETE /identity/v3/projects/2d8d04c44a994b6283d407737d942962 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:23.405681 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c195bcb-0e95-490c-94aa-82af686ba10d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.405967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c195bcb-0e95-490c-94aa-82af686ba10d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.406240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c195bcb-0e95-490c-94aa-82af686ba10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.426130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-0c195bcb-0e95-490c-94aa-82af686ba10d None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:23.428154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0c195bcb-0e95-490c-94aa-82af686ba10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:23.440284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1017/2042] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:23.452077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f1af1-cff8-4602-9c5b-921777736c0f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.453437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ca9f1af1-cff8-4602-9c5b-921777736c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:23.489189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1026/2043] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:23.496792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbbbb6-7d60-48b1-83bc-cbf1bcbed582 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.497106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbbbb6-7d60-48b1-83bc-cbf1bcbed582 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.497902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbbbb6-7d60-48b1-83bc-cbf1bcbed582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.514310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f1af1-cff8-4602-9c5b-921777736c0f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:23.518066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab6bc-8529-4298-917d-28d256234d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:23.518200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d2dbbbb6-7d60-48b1-83bc-cbf1bcbed582 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:23.518712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab6bc-8529-4298-917d-28d256234d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.518994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab6bc-8529-4298-917d-28d256234d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.519270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab6bc-8529-4298-917d-28d256234d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.519375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d2dbbbb6-7d60-48b1-83bc-cbf1bcbed582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:23.528179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1027/2044] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:23.529496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ab6bc-8529-4298-917d-28d256234d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:23.530939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ab6bc-8529-4298-917d-28d256234d50 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:23.532254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbbbb6-7d60-48b1-83bc-cbf1bcbed582 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.541211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1018/2045] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:23.546440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbbbb6-7d60-48b1-83bc-cbf1bcbed582 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:23.551507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5458-df57-4d94-a38d-a3e7c84ad213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:23.552339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5458-df57-4d94-a38d-a3e7c84ad213 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.552627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5458-df57-4d94-a38d-a3e7c84ad213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.552943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5458-df57-4d94-a38d-a3e7c84ad213 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.553604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb5458-df57-4d94-a38d-a3e7c84ad213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:23.554078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb5458-df57-4d94-a38d-a3e7c84ad213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:23.555712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb5458-df57-4d94-a38d-a3e7c84ad213 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:23.568951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1028/2046] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:58:23.573474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab6bc-8529-4298-917d-28d256234d50 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.584851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab6bc-8529-4298-917d-28d256234d50 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:23.589869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac7473-fa89-49eb-ad73-9abe3f82fe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:23.590675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac7473-fa89-49eb-ad73-9abe3f82fe3c admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.591046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac7473-fa89-49eb-ad73-9abe3f82fe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.591428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac7473-fa89-49eb-ad73-9abe3f82fe3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.592069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac7473-fa89-49eb-ad73-9abe3f82fe3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:23.594234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac7473-fa89-49eb-ad73-9abe3f82fe3c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:23.616351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:23.624454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1019/2047] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:23 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:23.628759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5458-df57-4d94-a38d-a3e7c84ad213 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.639959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5458-df57-4d94-a38d-a3e7c84ad213 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:23.710953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265892c-06d1-453f-9fce-74de22218013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:23.711578 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265892c-06d1-453f-9fce-74de22218013 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.711857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265892c-06d1-453f-9fce-74de22218013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.712162 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265892c-06d1-453f-9fce-74de22218013 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.712826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265892c-06d1-453f-9fce-74de22218013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:23.715013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265892c-06d1-453f-9fce-74de22218013 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:23.725812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:23.747552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1029/2048] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:23 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:23.749053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac7473-fa89-49eb-ad73-9abe3f82fe3c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.761293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac7473-fa89-49eb-ad73-9abe3f82fe3c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:23.766317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905237b1-68c5-4c84-8278-6baa58f0b857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:23.766857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905237b1-68c5-4c84-8278-6baa58f0b857 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.767136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905237b1-68c5-4c84-8278-6baa58f0b857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.767409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905237b1-68c5-4c84-8278-6baa58f0b857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.768054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905237b1-68c5-4c84-8278-6baa58f0b857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:23.769729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905237b1-68c5-4c84-8278-6baa58f0b857 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:23.780196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1020/2049] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:23.784237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265892c-06d1-453f-9fce-74de22218013 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.795127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265892c-06d1-453f-9fce-74de22218013 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:23.799949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a49eed-c071-4932-94a0-8a745e213acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:23.800587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a49eed-c071-4932-94a0-8a745e213acf admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.800897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a49eed-c071-4932-94a0-8a745e213acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.801170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a49eed-c071-4932-94a0-8a745e213acf admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.802338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a49eed-c071-4932-94a0-8a745e213acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:23.804075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a49eed-c071-4932-94a0-8a745e213acf admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:23.815482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1030/2050] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:23.820302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905237b1-68c5-4c84-8278-6baa58f0b857 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.830634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905237b1-68c5-4c84-8278-6baa58f0b857 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:23.835409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c4a1e-ac67-43ec-95fa-55e05e063acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:23.836091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c4a1e-ac67-43ec-95fa-55e05e063acd admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.836370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c4a1e-ac67-43ec-95fa-55e05e063acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.836643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c4a1e-ac67-43ec-95fa-55e05e063acd admin admin] PATH_INFO: `/v3/projects/2acf1f13cd334d97acb82b5fcbb73c2f/users/be64538f910d4e81a5470101bd5374fb/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.855051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c4a1e-ac67-43ec-95fa-55e05e063acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2acf1f13cd334d97acb82b5fcbb73c2f, user_id=be64538f910d4e81a5470101bd5374fb, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:23.859990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c4a1e-ac67-43ec-95fa-55e05e063acd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:23.885872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1021/2051] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:23 2022] PUT /identity/v3/projects/2acf1f13cd334d97acb82b5fcbb73c2f/users/be64538f910d4e81a5470101bd5374fb/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:23.896902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab3e1c-b261-4175-a739-f859dd6611a8 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:23.897489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab3e1c-b261-4175-a739-f859dd6611a8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:23.897894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab3e1c-b261-4175-a739-f859dd6611a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:23.914258 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c4a1e-ac67-43ec-95fa-55e05e063acd admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:23.918273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fa6c4a1e-ac67-43ec-95fa-55e05e063acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:23.943607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-8cab3e1c-b261-4175-a739-f859dd6611a8 None None] MFA Rules not processed for user `be64538f910d4e81a5470101bd5374fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:23.944894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8cab3e1c-b261-4175-a739-f859dd6611a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.030927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1031/2052] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:24.036220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c4a1e-ac67-43ec-95fa-55e05e063acd admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.037144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab3e1c-b261-4175-a739-f859dd6611a8 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.041735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.042411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.042665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.042932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] PATH_INFO: `/v3/projects/f1bcc1ab61f144f1a1622a71527a576c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.044098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1bcc1ab61f144f1a1622a71527a576c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.046079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.059209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab3e1c-b261-4175-a739-f859dd6611a8 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.064629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749dedc-fcb5-4dce-ad22-1be4163c5203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.065672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749dedc-fcb5-4dce-ad22-1be4163c5203 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.066032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749dedc-fcb5-4dce-ad22-1be4163c5203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.066373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749dedc-fcb5-4dce-ad22-1be4163c5203 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.067685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749dedc-fcb5-4dce-ad22-1be4163c5203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.069892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749dedc-fcb5-4dce-ad22-1be4163c5203 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.097206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1bcc1ab61f144f1a1622a71527a576c', 'request_id': 'req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.104237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1bcc1ab61f144f1a1622a71527a576c', 'request_id': 'req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.112012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1032/2053] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:24 2022] POST /identity/v3/projects => generated 381 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:24.113113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1bcc1ab61f144f1a1622a71527a576c', 'request_id': 'req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.116821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749dedc-fcb5-4dce-ad22-1be4163c5203 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.126256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] The token cache is being invalidate because project f1bcc1ab61f144f1a1622a71527a576c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:24.126543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.127196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.128697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1022/2054] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:23 2022] DELETE /identity/v3/projects/f1bcc1ab61f144f1a1622a71527a576c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:24.132640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.134181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.184217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749dedc-fcb5-4dce-ad22-1be4163c5203 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.189142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb307488-e33a-4042-ad78-6f8f0948e360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.189811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb307488-e33a-4042-ad78-6f8f0948e360 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.190091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb307488-e33a-4042-ad78-6f8f0948e360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.190362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb307488-e33a-4042-ad78-6f8f0948e360 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.191054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb307488-e33a-4042-ad78-6f8f0948e360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.193082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb307488-e33a-4042-ad78-6f8f0948e360 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.193245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:24.199595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272ff1d-f1d8-4680-b8c2-8ffa0bfdb5d4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.206478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.207244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.207727 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.207991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] PATH_INFO: `/v3/users/bce8045e9a6f4e0d89130ba3bf0ca8b8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.209295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce8045e9a6f4e0d89130ba3bf0ca8b8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.211216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.218006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1033/2055] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:24 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:24.219014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb307488-e33a-4042-ad78-6f8f0948e360 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.220613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cb307488-e33a-4042-ad78-6f8f0948e360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.233088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb307488-e33a-4042-ad78-6f8f0948e360 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.238323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a21f5-636d-4a38-9816-cce1ba97e6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.239863 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a21f5-636d-4a38-9816-cce1ba97e6ac admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.240199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a21f5-636d-4a38-9816-cce1ba97e6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.240473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a21f5-636d-4a38-9816-cce1ba97e6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.241163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a21f5-636d-4a38-9816-cce1ba97e6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.243100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a21f5-636d-4a38-9816-cce1ba97e6ac admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.255036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1034/2056] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:58:24.258901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce8045e9a6f4e0d89130ba3bf0ca8b8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.259314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a21f5-636d-4a38-9816-cce1ba97e6ac admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.266677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce8045e9a6f4e0d89130ba3bf0ca8b8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.267490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce8045e9a6f4e0d89130ba3bf0ca8b8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.269388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a21f5-636d-4a38-9816-cce1ba97e6ac admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.274511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3f447-09fa-4df3-add5-c8efaab0b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.274791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce8045e9a6f4e0d89130ba3bf0ca8b8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.275218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3f447-09fa-4df3-add5-c8efaab0b461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.275390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce8045e9a6f4e0d89130ba3bf0ca8b8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.275501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3f447-09fa-4df3-add5-c8efaab0b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.275801 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3f447-09fa-4df3-add5-c8efaab0b461 admin admin] PATH_INFO: `/v3/projects/f1233b3f0c854d40bab2586277952e20/users/ae6c42286c63469bb6615209786fdec9/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.292376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3f447-09fa-4df3-add5-c8efaab0b461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1233b3f0c854d40bab2586277952e20, user_id=ae6c42286c63469bb6615209786fdec9, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.295181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3f447-09fa-4df3-add5-c8efaab0b461 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.296674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1023/2057] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:24 2022] DELETE /identity/v3/users/bce8045e9a6f4e0d89130ba3bf0ca8b8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:24.310554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1035/2058] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:24 2022] PUT /identity/v3/projects/f1233b3f0c854d40bab2586277952e20/users/ae6c42286c63469bb6615209786fdec9/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:24.314678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.316208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.384110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e6016-c2bf-4f47-a7b1-b362971c2881 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.446640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18629ae9-fcae-4208-9946-96320a482590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.447343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629ae9-fcae-4208-9946-96320a482590 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.447557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629ae9-fcae-4208-9946-96320a482590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.448078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629ae9-fcae-4208-9946-96320a482590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.448750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18629ae9-fcae-4208-9946-96320a482590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.450400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18629ae9-fcae-4208-9946-96320a482590 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.461901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1024/2059] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:24.465860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3f447-09fa-4df3-add5-c8efaab0b461 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.472636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:24.478524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3f447-09fa-4df3-add5-c8efaab0b461 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.483821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42410953-9338-42fb-a06c-acfd697cf98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.484485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42410953-9338-42fb-a06c-acfd697cf98c admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.484822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42410953-9338-42fb-a06c-acfd697cf98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.485104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42410953-9338-42fb-a06c-acfd697cf98c admin admin] PATH_INFO: `/v3/projects/f1233b3f0c854d40bab2586277952e20/users/ae6c42286c63469bb6615209786fdec9/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.487033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42410953-9338-42fb-a06c-acfd697cf98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1233b3f0c854d40bab2586277952e20, user_id=ae6c42286c63469bb6615209786fdec9, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.490194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42410953-9338-42fb-a06c-acfd697cf98c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.501098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1036/2060] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:24 2022] PUT /identity/v3/projects/f1233b3f0c854d40bab2586277952e20/users/ae6c42286c63469bb6615209786fdec9/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:24.512469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f4da6-0338-4fcf-a947-9b91c1c4c9f7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.512761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f4da6-0338-4fcf-a947-9b91c1c4c9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.513068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f4da6-0338-4fcf-a947-9b91c1c4c9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.542809 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b85f4da6-0338-4fcf-a947-9b91c1c4c9f7 None None] MFA Rules not processed for user `ae6c42286c63469bb6615209786fdec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:24.543991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b85f4da6-0338-4fcf-a947-9b91c1c4c9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.549784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:24.582935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42410953-9338-42fb-a06c-acfd697cf98c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.584972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-42410953-9338-42fb-a06c-acfd697cf98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.635412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1025/2061] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:24.647738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f4da6-0338-4fcf-a947-9b91c1c4c9f7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.649310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b85f4da6-0338-4fcf-a947-9b91c1c4c9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.656429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42410953-9338-42fb-a06c-acfd697cf98c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.661321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.661949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.662227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.662495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] PATH_INFO: `/v3/projects/4b1d9e3cf9034b6ca2d016823fc4884f` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.663645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b1d9e3cf9034b6ca2d016823fc4884f)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.665590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.709729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f4da6-0338-4fcf-a947-9b91c1c4c9f7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.715652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b1d9e3cf9034b6ca2d016823fc4884f', 'request_id': 'req-15a6ed4d-f391-460c-9358-065683e7c8ce'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.721425 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b1d9e3cf9034b6ca2d016823fc4884f', 'request_id': 'req-15a6ed4d-f391-460c-9358-065683e7c8ce'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.729730 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b1d9e3cf9034b6ca2d016823fc4884f', 'request_id': 'req-15a6ed4d-f391-460c-9358-065683e7c8ce'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.742139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] The token cache is being invalidate because project 4b1d9e3cf9034b6ca2d016823fc4884f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:24.742428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.743084 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-15a6ed4d-f391-460c-9358-065683e7c8ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:24.744597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1037/2062] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:24 2022] DELETE /identity/v3/projects/4b1d9e3cf9034b6ca2d016823fc4884f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:24.749499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6ea4b-9bf8-4e74-b8d6-c3d88f18e2b6 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.749898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6ea4b-9bf8-4e74-b8d6-c3d88f18e2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.750180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6ea4b-9bf8-4e74-b8d6-c3d88f18e2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.771307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cc551-b519-454a-bb98-d6dfdb464cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.771894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-92d6ea4b-9bf8-4e74-b8d6-c3d88f18e2b6 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:24.772137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cc551-b519-454a-bb98-d6dfdb464cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.772357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cc551-b519-454a-bb98-d6dfdb464cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.772626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cc551-b519-454a-bb98-d6dfdb464cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.773088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-92d6ea4b-9bf8-4e74-b8d6-c3d88f18e2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.774495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ec6cc551-b519-454a-bb98-d6dfdb464cb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.846263 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1038/2063] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:24.846451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cc551-b519-454a-bb98-d6dfdb464cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.846451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cc551-b519-454a-bb98-d6dfdb464cb0 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.847657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dbe59-5494-46bd-8485-d13d10cbc1e2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.847944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dbe59-5494-46bd-8485-d13d10cbc1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.848238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dbe59-5494-46bd-8485-d13d10cbc1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.863582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1026/2064] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:24.870242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-375dbe59-5494-46bd-8485-d13d10cbc1e2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:24.871468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-375dbe59-5494-46bd-8485-d13d10cbc1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:24.884413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1039/2065] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:24.898173 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cc551-b519-454a-bb98-d6dfdb464cb0 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:24.932756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cc551-b519-454a-bb98-d6dfdb464cb0 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:24.950964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567e151-761b-4252-af44-914b23b4d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:24.953669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567e151-761b-4252-af44-914b23b4d43b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:24.955350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567e151-761b-4252-af44-914b23b4d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:24.956438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567e151-761b-4252-af44-914b23b4d43b admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:24.960409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567e151-761b-4252-af44-914b23b4d43b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:24.961562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567e151-761b-4252-af44-914b23b4d43b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:24.968282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567e151-761b-4252-af44-914b23b4d43b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:24.977675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:24.996016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1027/2066] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:58:24.996711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dbe59-5494-46bd-8485-d13d10cbc1e2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.011689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dbe59-5494-46bd-8485-d13d10cbc1e2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.019031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef9e5e-9d29-4d1e-be90-88e0000dfff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.019813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9e5e-9d29-4d1e-be90-88e0000dfff7 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.021465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9e5e-9d29-4d1e-be90-88e0000dfff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.021750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9e5e-9d29-4d1e-be90-88e0000dfff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.022423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef9e5e-9d29-4d1e-be90-88e0000dfff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.026510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef9e5e-9d29-4d1e-be90-88e0000dfff7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.046958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:25.081481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1040/2067] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:24 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:25.084187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567e151-761b-4252-af44-914b23b4d43b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.100956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567e151-761b-4252-af44-914b23b4d43b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.224286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4f726-e552-4f5d-9a1f-77103cde7848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.225267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4f726-e552-4f5d-9a1f-77103cde7848 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.225646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4f726-e552-4f5d-9a1f-77103cde7848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.226179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4f726-e552-4f5d-9a1f-77103cde7848 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.226917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4f726-e552-4f5d-9a1f-77103cde7848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.235612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4f726-e552-4f5d-9a1f-77103cde7848 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.269762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1028/2068] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:25 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:25.275583 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef9e5e-9d29-4d1e-be90-88e0000dfff7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.293697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef9e5e-9d29-4d1e-be90-88e0000dfff7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.306120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6f94f-68a1-406f-8095-88416a72b765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.308129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6f94f-68a1-406f-8095-88416a72b765 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.309016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6f94f-68a1-406f-8095-88416a72b765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.309301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6f94f-68a1-406f-8095-88416a72b765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.310107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6f94f-68a1-406f-8095-88416a72b765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.311879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6f94f-68a1-406f-8095-88416a72b765 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.351954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1041/2069] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:25.358054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4f726-e552-4f5d-9a1f-77103cde7848 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.377876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4f726-e552-4f5d-9a1f-77103cde7848 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.387187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341a4d0-5377-41b8-ae08-a9db5a7f37ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.387883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341a4d0-5377-41b8-ae08-a9db5a7f37ed admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.388232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341a4d0-5377-41b8-ae08-a9db5a7f37ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.388656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341a4d0-5377-41b8-ae08-a9db5a7f37ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.389360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341a4d0-5377-41b8-ae08-a9db5a7f37ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.390681 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6f94f-68a1-406f-8095-88416a72b765 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.404011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341a4d0-5377-41b8-ae08-a9db5a7f37ed admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.420894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1029/2070] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:25.430436 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341a4d0-5377-41b8-ae08-a9db5a7f37ed admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.449235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341a4d0-5377-41b8-ae08-a9db5a7f37ed admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.456065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a5628-3f50-4106-b6c5-f00af40cb0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.456736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a5628-3f50-4106-b6c5-f00af40cb0e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.457053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a5628-3f50-4106-b6c5-f00af40cb0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.457329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a5628-3f50-4106-b6c5-f00af40cb0e0 admin admin] PATH_INFO: `/v3/projects/2c853fd25a4241998f2d6c6af65df780/users/baf3f8f4e8f740c5b9693e92fd074b3f/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.474550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a5628-3f50-4106-b6c5-f00af40cb0e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c853fd25a4241998f2d6c6af65df780, user_id=baf3f8f4e8f740c5b9693e92fd074b3f, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.477298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a5628-3f50-4106-b6c5-f00af40cb0e0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.477757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6f94f-68a1-406f-8095-88416a72b765 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.483201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923525-a178-46df-a70b-0ebbd2022bcd tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6c42286c63469bb6615209786fdec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1233b3f0c854d40bab2586277952e20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.484162 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923525-a178-46df-a70b-0ebbd2022bcd tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.484162 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923525-a178-46df-a70b-0ebbd2022bcd tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.484472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923525-a178-46df-a70b-0ebbd2022bcd tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.485305 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67923525-a178-46df-a70b-0ebbd2022bcd tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:25.485623 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67923525-a178-46df-a70b-0ebbd2022bcd tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.487346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67923525-a178-46df-a70b-0ebbd2022bcd tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.496850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1030/2071] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:25 2022] PUT /identity/v3/projects/2c853fd25a4241998f2d6c6af65df780/users/baf3f8f4e8f740c5b9693e92fd074b3f/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:25.501819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78ec8c-aa10-409b-9253-b05016e8182c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.502326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78ec8c-aa10-409b-9253-b05016e8182c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.502785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78ec8c-aa10-409b-9253-b05016e8182c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.505579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1042/2072] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:58:25.509738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923525-a178-46df-a70b-0ebbd2022bcd tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.522005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923525-a178-46df-a70b-0ebbd2022bcd tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.525556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ab78ec8c-aa10-409b-9253-b05016e8182c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:25.527119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ab78ec8c-aa10-409b-9253-b05016e8182c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:25.594653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048722a-80b1-4389-bb80-cac6ea7290d4 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6c42286c63469bb6615209786fdec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1233b3f0c854d40bab2586277952e20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.595293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048722a-80b1-4389-bb80-cac6ea7290d4 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.595570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048722a-80b1-4389-bb80-cac6ea7290d4 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.595848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048722a-80b1-4389-bb80-cac6ea7290d4 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.596775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048722a-80b1-4389-bb80-cac6ea7290d4 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.598990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048722a-80b1-4389-bb80-cac6ea7290d4 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.607534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1031/2073] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:25.621036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78ec8c-aa10-409b-9253-b05016e8182c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.622754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ab78ec8c-aa10-409b-9253-b05016e8182c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:25.630030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1043/2074] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:25 2022] POST /identity/v3/projects => generated 355 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:25.635150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048722a-80b1-4389-bb80-cac6ea7290d4 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.693741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78ec8c-aa10-409b-9253-b05016e8182c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.699464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:25.715470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048722a-80b1-4389-bb80-cac6ea7290d4 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.721461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaf8f4-dc8a-44a3-844a-b51752c82557 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6c42286c63469bb6615209786fdec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1233b3f0c854d40bab2586277952e20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.722182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaf8f4-dc8a-44a3-844a-b51752c82557 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.722568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaf8f4-dc8a-44a3-844a-b51752c82557 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.722945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaf8f4-dc8a-44a3-844a-b51752c82557 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.723733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaf8f4-dc8a-44a3-844a-b51752c82557 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:25.724169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaf8f4-dc8a-44a3-844a-b51752c82557 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.726053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaf8f4-dc8a-44a3-844a-b51752c82557 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.729492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:25.743018 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1044/2075] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:58:25.763806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe76250-7967-4bca-9c02-cbae1845abf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.764528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe76250-7967-4bca-9c02-cbae1845abf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.764824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe76250-7967-4bca-9c02-cbae1845abf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.765244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe76250-7967-4bca-9c02-cbae1845abf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.781339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaf8f4-dc8a-44a3-844a-b51752c82557 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:25.791524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaf8f4-dc8a-44a3-844a-b51752c82557 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:25.796160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae6c42286c63469bb6615209786fdec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1233b3f0c854d40bab2586277952e20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:25.796996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:25.797108 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:25.797360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] PATH_INFO: `/v3/projects/411a7f4227d7477c87c9ace0a92081e3` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:25.798689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Authorizing `identity:delete_project(project_id=411a7f4227d7477c87c9ace0a92081e3)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.800611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.835041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe76250-7967-4bca-9c02-cbae1845abf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:25.836652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe76250-7967-4bca-9c02-cbae1845abf7 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:25.848896 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1032/2076] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:25.860829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '411a7f4227d7477c87c9ace0a92081e3', 'request_id': 'req-f9542c48-858f-4d7f-8553-86621c08c038'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:25.867859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '411a7f4227d7477c87c9ace0a92081e3', 'request_id': 'req-f9542c48-858f-4d7f-8553-86621c08c038'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:25.877818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '411a7f4227d7477c87c9ace0a92081e3', 'request_id': 'req-f9542c48-858f-4d7f-8553-86621c08c038'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:25.893053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] The token cache is being invalidate because project 411a7f4227d7477c87c9ace0a92081e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:25.893357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:25.894124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f9542c48-858f-4d7f-8553-86621c08c038 tempest-VolumeQuotaClassesTest-2023036619 tempest-VolumeQuotaClassesTest-2023036619-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:25.896593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1045/2077] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:25 2022] DELETE /identity/v3/projects/411a7f4227d7477c87c9ace0a92081e3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:26.124316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe76250-7967-4bca-9c02-cbae1845abf7 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:26.125887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cfe76250-7967-4bca-9c02-cbae1845abf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:26.140069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe76250-7967-4bca-9c02-cbae1845abf7 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:26.202109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:26.210411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:26.210704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:26.211154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] PATH_INFO: `/v3/users/be64538f910d4e81a5470101bd5374fb` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:26.213466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be64538f910d4e81a5470101bd5374fb)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:26.215886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:26.287194 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be64538f910d4e81a5470101bd5374fb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:26.296103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be64538f910d4e81a5470101bd5374fb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:26.296775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be64538f910d4e81a5470101bd5374fb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:26.302929 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:26.307810 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be64538f910d4e81a5470101bd5374fb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:26.309536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be64538f910d4e81a5470101bd5374fb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:26.340890 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1033/2078] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:26 2022] DELETE /identity/v3/users/be64538f910d4e81a5470101bd5374fb => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:26.344919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f56dd-efde-458a-b461-a39f0b24d741 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:26.345373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f56dd-efde-458a-b461-a39f0b24d741 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:26.345463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f56dd-efde-458a-b461-a39f0b24d741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:26.369189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-068f56dd-efde-458a-b461-a39f0b24d741 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:26.370331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-068f56dd-efde-458a-b461-a39f0b24d741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:26.453567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1046/2079] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:26.466611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:26.469408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:26.562786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba69c-b829-4074-a279-2c8da649062b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:26.567125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1256839-3caa-4409-8b66-35d65985e001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:26.568735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1256839-3caa-4409-8b66-35d65985e001 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:26.569205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1256839-3caa-4409-8b66-35d65985e001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:26.570316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1256839-3caa-4409-8b66-35d65985e001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:26.585472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1256839-3caa-4409-8b66-35d65985e001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:26.587035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1256839-3caa-4409-8b66-35d65985e001 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:26.599876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1034/2080] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:26.665899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b42e1-7129-40db-95c3-573cf38fe456 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:26.666658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b42e1-7129-40db-95c3-573cf38fe456 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:26.666658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b42e1-7129-40db-95c3-573cf38fe456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:26.700128 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-191b42e1-7129-40db-95c3-573cf38fe456 None None] MFA Rules not processed for user `baf3f8f4e8f740c5b9693e92fd074b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:26.701434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-191b42e1-7129-40db-95c3-573cf38fe456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:26.802168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1047/2081] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:26.806572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1256839-3caa-4409-8b66-35d65985e001 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:26.809122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d1256839-3caa-4409-8b66-35d65985e001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:26.830405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1256839-3caa-4409-8b66-35d65985e001 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:26.836379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43627a94-3d2e-40ba-9910-73b3bc05fc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:26.837278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43627a94-3d2e-40ba-9910-73b3bc05fc04 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:26.837664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43627a94-3d2e-40ba-9910-73b3bc05fc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:26.838340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43627a94-3d2e-40ba-9910-73b3bc05fc04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:26.839082 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43627a94-3d2e-40ba-9910-73b3bc05fc04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:26.841475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43627a94-3d2e-40ba-9910-73b3bc05fc04 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:26.877434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1035/2082] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:26 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:26.878655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b42e1-7129-40db-95c3-573cf38fe456 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:26.906502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b42e1-7129-40db-95c3-573cf38fe456 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:26.914896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:27.024758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8cfb3-330c-4b35-b9fc-9f33e206fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:27.025405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8cfb3-330c-4b35-b9fc-9f33e206fb07 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:27.025685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8cfb3-330c-4b35-b9fc-9f33e206fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:27.025953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8cfb3-330c-4b35-b9fc-9f33e206fb07 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:27.026907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8cfb3-330c-4b35-b9fc-9f33e206fb07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:27.028580 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8cfb3-330c-4b35-b9fc-9f33e206fb07 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:27.062977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1048/2083] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:26 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:27.064479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43627a94-3d2e-40ba-9910-73b3bc05fc04 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:27.076848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43627a94-3d2e-40ba-9910-73b3bc05fc04 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:27.082211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cc265-365e-4d3b-84e6-39071f74ba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:27.083136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cc265-365e-4d3b-84e6-39071f74ba74 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:27.083326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cc265-365e-4d3b-84e6-39071f74ba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:27.083574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cc265-365e-4d3b-84e6-39071f74ba74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:27.084248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5cc265-365e-4d3b-84e6-39071f74ba74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:27.086226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5cc265-365e-4d3b-84e6-39071f74ba74 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:27.101653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1036/2084] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:58:27.105014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8cfb3-330c-4b35-b9fc-9f33e206fb07 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:27.122576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8cfb3-330c-4b35-b9fc-9f33e206fb07 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:27.128111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3dc1b-e2b6-4c26-a870-a9e0e7daa5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:27.128826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3dc1b-e2b6-4c26-a870-a9e0e7daa5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:27.129178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3dc1b-e2b6-4c26-a870-a9e0e7daa5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:27.129521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3dc1b-e2b6-4c26-a870-a9e0e7daa5fd admin admin] PATH_INFO: `/v3/projects/8cdc953fcc854694a04ab4da5c366fb2/users/6907bc47b2e44dac8fa3253ae98aad8e/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:27.145710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3dc1b-e2b6-4c26-a870-a9e0e7daa5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cdc953fcc854694a04ab4da5c366fb2, user_id=6907bc47b2e44dac8fa3253ae98aad8e, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:27.148406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3dc1b-e2b6-4c26-a870-a9e0e7daa5fd admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:27.164177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1049/2085] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:27 2022] PUT /identity/v3/projects/8cdc953fcc854694a04ab4da5c366fb2/users/6907bc47b2e44dac8fa3253ae98aad8e/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:27.168329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cc265-365e-4d3b-84e6-39071f74ba74 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:27.181168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3dc1b-e2b6-4c26-a870-a9e0e7daa5fd admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:27.182633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cc265-365e-4d3b-84e6-39071f74ba74 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:27.185654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cfe3dc1b-e2b6-4c26-a870-a9e0e7daa5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:27.253092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae21fa-5912-4e46-9096-2561f7d42e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:27.253765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae21fa-5912-4e46-9096-2561f7d42e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:27.254050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae21fa-5912-4e46-9096-2561f7d42e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:27.254365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae21fa-5912-4e46-9096-2561f7d42e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:27.255281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae21fa-5912-4e46-9096-2561f7d42e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:27.256994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae21fa-5912-4e46-9096-2561f7d42e71 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:27.268237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3dc1b-e2b6-4c26-a870-a9e0e7daa5fd admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:27.268435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1037/2086] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:58:27.272701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae21fa-5912-4e46-9096-2561f7d42e71 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:27.273758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:27.274622 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:27.275102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:27.275609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] PATH_INFO: `/v3/projects/2acf1f13cd334d97acb82b5fcbb73c2f` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:27.276960 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2acf1f13cd334d97acb82b5fcbb73c2f)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:27.279092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:27.283149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae21fa-5912-4e46-9096-2561f7d42e71 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:27.289460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96054c-f3c4-4a25-b925-e708274b2653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:27.290118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96054c-f3c4-4a25-b925-e708274b2653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:27.290448 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96054c-f3c4-4a25-b925-e708274b2653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:27.290716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96054c-f3c4-4a25-b925-e708274b2653 admin admin] PATH_INFO: `/v3/projects/8cdc953fcc854694a04ab4da5c366fb2/users/6907bc47b2e44dac8fa3253ae98aad8e/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:27.293261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96054c-f3c4-4a25-b925-e708274b2653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cdc953fcc854694a04ab4da5c366fb2, user_id=6907bc47b2e44dac8fa3253ae98aad8e, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:27.296264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96054c-f3c4-4a25-b925-e708274b2653 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:27.309956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1038/2087] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:27 2022] PUT /identity/v3/projects/8cdc953fcc854694a04ab4da5c366fb2/users/6907bc47b2e44dac8fa3253ae98aad8e/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:27.349417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2acf1f13cd334d97acb82b5fcbb73c2f', 'request_id': 'req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.356325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2acf1f13cd334d97acb82b5fcbb73c2f', 'request_id': 'req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.365124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2acf1f13cd334d97acb82b5fcbb73c2f', 'request_id': 'req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.382070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] The token cache is being invalidate because project 2acf1f13cd334d97acb82b5fcbb73c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:27.382370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.383073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.384958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1050/2088] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:27 2022] DELETE /identity/v3/projects/2acf1f13cd334d97acb82b5fcbb73c2f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:27.388664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96054c-f3c4-4a25-b925-e708274b2653 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:27.391047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fc96054c-f3c4-4a25-b925-e708274b2653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:27.431237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:27.475139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96054c-f3c4-4a25-b925-e708274b2653 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:27.559869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:27.560445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:27.560825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:27.561185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] PATH_INFO: `/v3/users/ae6c42286c63469bb6615209786fdec9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:27.562641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6c42286c63469bb6615209786fdec9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:27.564920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:27.631545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6c42286c63469bb6615209786fdec9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.639767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6c42286c63469bb6615209786fdec9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.641041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6c42286c63469bb6615209786fdec9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.654380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6c42286c63469bb6615209786fdec9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.655314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ab3aae2-2b2e-46b1-b6b3-4f26f39aad63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6c42286c63469bb6615209786fdec9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:27.700289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1039/2089] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:27 2022] DELETE /identity/v3/users/ae6c42286c63469bb6615209786fdec9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:28.552994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:28.560353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:28.567480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc4fd9-d0d2-4ba6-938e-3bb577bd732b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:28.578988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963aa8-eb56-4fc3-9719-6d7015fdb01e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:28.579819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963aa8-eb56-4fc3-9719-6d7015fdb01e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:28.580102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963aa8-eb56-4fc3-9719-6d7015fdb01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:28.616550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-12963aa8-eb56-4fc3-9719-6d7015fdb01e None None] MFA Rules not processed for user `6907bc47b2e44dac8fa3253ae98aad8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:28.618001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-12963aa8-eb56-4fc3-9719-6d7015fdb01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:28.655285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:28.656275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:28.656563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:28.657346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] PATH_INFO: `/v3/projects/f1233b3f0c854d40bab2586277952e20` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:28.658831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1233b3f0c854d40bab2586277952e20)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:28.660762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:28.672556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:28.709790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1040/2090] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:28.720874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963aa8-eb56-4fc3-9719-6d7015fdb01e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:28.723588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-12963aa8-eb56-4fc3-9719-6d7015fdb01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:28.724995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1233b3f0c854d40bab2586277952e20', 'request_id': 'req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:28.731912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1233b3f0c854d40bab2586277952e20', 'request_id': 'req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:28.742429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1233b3f0c854d40bab2586277952e20', 'request_id': 'req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:28.756082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] The token cache is being invalidate because project f1233b3f0c854d40bab2586277952e20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:28.756431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:28.757111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a2f384a-af0c-4bb4-8bef-ae7b75051d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:28.762078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1051/2091] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:28 2022] DELETE /identity/v3/projects/f1233b3f0c854d40bab2586277952e20 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:28.763297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589759e3-1660-4acb-9cf6-d8ed584e0d7e None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:28.763575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589759e3-1660-4acb-9cf6-d8ed584e0d7e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:28.763852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589759e3-1660-4acb-9cf6-d8ed584e0d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:28.784005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-589759e3-1660-4acb-9cf6-d8ed584e0d7e None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:28.785832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-589759e3-1660-4acb-9cf6-d8ed584e0d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:28.787924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963aa8-eb56-4fc3-9719-6d7015fdb01e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:28.840251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535757e-81fe-4357-9a3a-da845ad126a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:28.841045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535757e-81fe-4357-9a3a-da845ad126a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:28.841327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535757e-81fe-4357-9a3a-da845ad126a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:28.841600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535757e-81fe-4357-9a3a-da845ad126a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:28.843584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0535757e-81fe-4357-9a3a-da845ad126a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:28.858650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1052/2092] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:28.861286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f885c9a-3e71-4fb0-a887-efefcf0f3bd7 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:28.861817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f885c9a-3e71-4fb0-a887-efefcf0f3bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:28.862333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f885c9a-3e71-4fb0-a887-efefcf0f3bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:28.888946 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-1f885c9a-3e71-4fb0-a887-efefcf0f3bd7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:28.890272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-1f885c9a-3e71-4fb0-a887-efefcf0f3bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:28.903755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1053/2093] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:28.904745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f885c9a-3e71-4fb0-a887-efefcf0f3bd7 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:28.914900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f885c9a-3e71-4fb0-a887-efefcf0f3bd7 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:28.916321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535757e-81fe-4357-9a3a-da845ad126a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:28.918699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535757e-81fe-4357-9a3a-da845ad126a3 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:28.919568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca53d2-97c8-422b-b411-7b3a48b7762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:28.920184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca53d2-97c8-422b-b411-7b3a48b7762a admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:28.920464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca53d2-97c8-422b-b411-7b3a48b7762a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:28.920737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca53d2-97c8-422b-b411-7b3a48b7762a admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:28.921417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca53d2-97c8-422b-b411-7b3a48b7762a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:28.921722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca53d2-97c8-422b-b411-7b3a48b7762a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:28.923335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca53d2-97c8-422b-b411-7b3a48b7762a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:28.931020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1041/2094] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:28.940333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1054/2095] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:58:28.941184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535757e-81fe-4357-9a3a-da845ad126a3 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:28.954209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535757e-81fe-4357-9a3a-da845ad126a3 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:28.959401 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fb043-dc3e-4857-8a09-6f62cea346ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:28.960016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fb043-dc3e-4857-8a09-6f62cea346ca admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:28.960325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fb043-dc3e-4857-8a09-6f62cea346ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:28.960603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fb043-dc3e-4857-8a09-6f62cea346ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:28.961265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fb043-dc3e-4857-8a09-6f62cea346ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:28.963206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fb043-dc3e-4857-8a09-6f62cea346ca admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:28.997656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1042/2096] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:28 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:29.001744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca53d2-97c8-422b-b411-7b3a48b7762a admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.012378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca53d2-97c8-422b-b411-7b3a48b7762a admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.031496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:29.092136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64015f9c-5ad6-49cc-9a37-2f11ee93c615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.092764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64015f9c-5ad6-49cc-9a37-2f11ee93c615 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.093461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64015f9c-5ad6-49cc-9a37-2f11ee93c615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.093750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64015f9c-5ad6-49cc-9a37-2f11ee93c615 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.094584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64015f9c-5ad6-49cc-9a37-2f11ee93c615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.097002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64015f9c-5ad6-49cc-9a37-2f11ee93c615 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.124306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1055/2097] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:29 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:29.125531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fb043-dc3e-4857-8a09-6f62cea346ca admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.138211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fb043-dc3e-4857-8a09-6f62cea346ca admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.145209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0dadd-fc38-42ea-967b-f376fc834873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.145916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0dadd-fc38-42ea-967b-f376fc834873 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.146196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0dadd-fc38-42ea-967b-f376fc834873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.146478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0dadd-fc38-42ea-967b-f376fc834873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.147122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0dadd-fc38-42ea-967b-f376fc834873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.148768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0dadd-fc38-42ea-967b-f376fc834873 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.164981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1043/2098] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:29.168648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64015f9c-5ad6-49cc-9a37-2f11ee93c615 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.180938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64015f9c-5ad6-49cc-9a37-2f11ee93c615 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.186770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6dd6b3-19c7-4f5e-a048-3254e966dd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.187416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6dd6b3-19c7-4f5e-a048-3254e966dd28 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.187696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6dd6b3-19c7-4f5e-a048-3254e966dd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.187972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6dd6b3-19c7-4f5e-a048-3254e966dd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.188618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6dd6b3-19c7-4f5e-a048-3254e966dd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.190319 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6dd6b3-19c7-4f5e-a048-3254e966dd28 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.203753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1056/2099] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:29.207705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0dadd-fc38-42ea-967b-f376fc834873 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.225019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0dadd-fc38-42ea-967b-f376fc834873 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.249270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19696a0f-dd23-4d03-8f9e-aa39a66bd2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.250112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19696a0f-dd23-4d03-8f9e-aa39a66bd2ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.250665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19696a0f-dd23-4d03-8f9e-aa39a66bd2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.251112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19696a0f-dd23-4d03-8f9e-aa39a66bd2ca admin admin] PATH_INFO: `/v3/projects/ef3f991ce37a447f8945326d323f5e20/users/312e736fc3e64a0c9ef837368174eddb/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.304846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19696a0f-dd23-4d03-8f9e-aa39a66bd2ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3f991ce37a447f8945326d323f5e20, user_id=312e736fc3e64a0c9ef837368174eddb, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.308322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19696a0f-dd23-4d03-8f9e-aa39a66bd2ca admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.311317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:29.326677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1044/2100] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:29 2022] PUT /identity/v3/projects/ef3f991ce37a447f8945326d323f5e20/users/312e736fc3e64a0c9ef837368174eddb/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:29.339413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079e327-f462-405c-8dee-402c7029f303 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.339793 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079e327-f462-405c-8dee-402c7029f303 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.340099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079e327-f462-405c-8dee-402c7029f303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.374495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e079e327-f462-405c-8dee-402c7029f303 None None] MFA Rules not processed for user `312e736fc3e64a0c9ef837368174eddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:29.375791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e079e327-f462-405c-8dee-402c7029f303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:29.446358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1057/2101] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:29.451387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19696a0f-dd23-4d03-8f9e-aa39a66bd2ca admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.513179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19696a0f-dd23-4d03-8f9e-aa39a66bd2ca admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.517812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197babc-65f0-41dc-bb38-4fab6ab40c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.518416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197babc-65f0-41dc-bb38-4fab6ab40c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.518700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197babc-65f0-41dc-bb38-4fab6ab40c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.518970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197babc-65f0-41dc-bb38-4fab6ab40c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.519607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197babc-65f0-41dc-bb38-4fab6ab40c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.521804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197babc-65f0-41dc-bb38-4fab6ab40c15 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.553629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1045/2102] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:29 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:29.557619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079e327-f462-405c-8dee-402c7029f303 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.568603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079e327-f462-405c-8dee-402c7029f303 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.625435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85951-ccbf-4b87-b3f3-ed6214d593cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.626196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85951-ccbf-4b87-b3f3-ed6214d593cb admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.626484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85951-ccbf-4b87-b3f3-ed6214d593cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.626760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85951-ccbf-4b87-b3f3-ed6214d593cb admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.627389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc85951-ccbf-4b87-b3f3-ed6214d593cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.629407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc85951-ccbf-4b87-b3f3-ed6214d593cb admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.650977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1058/2103] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:29 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:29.655272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197babc-65f0-41dc-bb38-4fab6ab40c15 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.667083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197babc-65f0-41dc-bb38-4fab6ab40c15 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.671721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f94d7-1d4d-4dc9-aa1a-bdb28ba3a7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.672313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f94d7-1d4d-4dc9-aa1a-bdb28ba3a7af admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.672594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f94d7-1d4d-4dc9-aa1a-bdb28ba3a7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.672908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f94d7-1d4d-4dc9-aa1a-bdb28ba3a7af admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.673552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f94d7-1d4d-4dc9-aa1a-bdb28ba3a7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.675155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f94d7-1d4d-4dc9-aa1a-bdb28ba3a7af admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.685898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1046/2104] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:29.689929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85951-ccbf-4b87-b3f3-ed6214d593cb admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.700242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85951-ccbf-4b87-b3f3-ed6214d593cb admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.704852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86b641-ec19-40da-8792-5da243ab3c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.705502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86b641-ec19-40da-8792-5da243ab3c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.705782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86b641-ec19-40da-8792-5da243ab3c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.706051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86b641-ec19-40da-8792-5da243ab3c85 admin admin] PATH_INFO: `/v3/projects/f035b78f4a824f349bacd85f8b90735c/users/2cee7819eff9425f932f2216c799c2db/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.717937 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:29.722505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86b641-ec19-40da-8792-5da243ab3c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f035b78f4a824f349bacd85f8b90735c, user_id=2cee7819eff9425f932f2216c799c2db, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.726571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86b641-ec19-40da-8792-5da243ab3c85 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.739242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1059/2105] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:29 2022] PUT /identity/v3/projects/f035b78f4a824f349bacd85f8b90735c/users/2cee7819eff9425f932f2216c799c2db/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:29.743117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f94d7-1d4d-4dc9-aa1a-bdb28ba3a7af admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.753012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f94d7-1d4d-4dc9-aa1a-bdb28ba3a7af admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.807905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368a6d5-3524-494f-8971-7cbfdb4750b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.808432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368a6d5-3524-494f-8971-7cbfdb4750b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.808719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368a6d5-3524-494f-8971-7cbfdb4750b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.809021 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368a6d5-3524-494f-8971-7cbfdb4750b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.809773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368a6d5-3524-494f-8971-7cbfdb4750b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.811424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368a6d5-3524-494f-8971-7cbfdb4750b7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.821297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1047/2106] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:29.825872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86b641-ec19-40da-8792-5da243ab3c85 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.836903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86b641-ec19-40da-8792-5da243ab3c85 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:29.841498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3347a24-eaa4-401a-9c4c-e48dfaa1a627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:29.842146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3347a24-eaa4-401a-9c4c-e48dfaa1a627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.842430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3347a24-eaa4-401a-9c4c-e48dfaa1a627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.842708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3347a24-eaa4-401a-9c4c-e48dfaa1a627 admin admin] PATH_INFO: `/v3/projects/f035b78f4a824f349bacd85f8b90735c/users/2cee7819eff9425f932f2216c799c2db/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.844648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3347a24-eaa4-401a-9c4c-e48dfaa1a627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f035b78f4a824f349bacd85f8b90735c, user_id=2cee7819eff9425f932f2216c799c2db, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:29.847366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3347a24-eaa4-401a-9c4c-e48dfaa1a627 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:29.858366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1060/2107] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:29 2022] PUT /identity/v3/projects/f035b78f4a824f349bacd85f8b90735c/users/2cee7819eff9425f932f2216c799c2db/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:29.868517 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8e4d6-8181-4ff9-9d65-f749cc1bde6e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:29.868951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8e4d6-8181-4ff9-9d65-f749cc1bde6e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:29.869252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8e4d6-8181-4ff9-9d65-f749cc1bde6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:29.897127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0ae8e4d6-8181-4ff9-9d65-f749cc1bde6e None None] MFA Rules not processed for user `2cee7819eff9425f932f2216c799c2db`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:29.898353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0ae8e4d6-8181-4ff9-9d65-f749cc1bde6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:29.947209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:29.967502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1048/2108] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:29.979323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3347a24-eaa4-401a-9c4c-e48dfaa1a627 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:29.981027 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a3347a24-eaa4-401a-9c4c-e48dfaa1a627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:30.046457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3347a24-eaa4-401a-9c4c-e48dfaa1a627 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:30.051011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb795ab8-0870-479c-93a2-b8a7f6ec234e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:30.051887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb795ab8-0870-479c-93a2-b8a7f6ec234e service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:30.052171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb795ab8-0870-479c-93a2-b8a7f6ec234e service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:30.052447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb795ab8-0870-479c-93a2-b8a7f6ec234e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:30.063178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb795ab8-0870-479c-93a2-b8a7f6ec234e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:30.065109 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb795ab8-0870-479c-93a2-b8a7f6ec234e service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:30.077460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1061/2109] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:30.205763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8e4d6-8181-4ff9-9d65-f749cc1bde6e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:30.216738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8e4d6-8181-4ff9-9d65-f749cc1bde6e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:30.221197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d17e4-3d4c-42d9-8df5-db69d26290f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:30.221724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d17e4-3d4c-42d9-8df5-db69d26290f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:30.222003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d17e4-3d4c-42d9-8df5-db69d26290f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:30.222273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d17e4-3d4c-42d9-8df5-db69d26290f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:30.292642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d17e4-3d4c-42d9-8df5-db69d26290f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:30.294556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d17e4-3d4c-42d9-8df5-db69d26290f5 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:30.306249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1049/2110] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:30.387844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb795ab8-0870-479c-93a2-b8a7f6ec234e service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:30.389740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fb795ab8-0870-479c-93a2-b8a7f6ec234e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:30.439187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb795ab8-0870-479c-93a2-b8a7f6ec234e service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:30.453943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbcb9a-d272-4bbc-9162-426d5589369e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:30.455123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbcb9a-d272-4bbc-9162-426d5589369e service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:30.455574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbcb9a-d272-4bbc-9162-426d5589369e service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:30.456172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbcb9a-d272-4bbc-9162-426d5589369e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:30.459355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fecbcb9a-d272-4bbc-9162-426d5589369e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:30.607465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbcb9a-d272-4bbc-9162-426d5589369e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:30.609317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbcb9a-d272-4bbc-9162-426d5589369e service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:30.624614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1062/2111] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:33.244514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d17e4-3d4c-42d9-8df5-db69d26290f5 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:33.309387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d17e4-3d4c-42d9-8df5-db69d26290f5 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:33.317634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:33.318319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:33.318602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:33.318876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] PATH_INFO: `/v3/users/312e736fc3e64a0c9ef837368174eddb` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:33.320054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] RBAC: Authorizing `identity:delete_user(user_id=312e736fc3e64a0c9ef837368174eddb)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:33.322039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:33.372394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312e736fc3e64a0c9ef837368174eddb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:33.380055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312e736fc3e64a0c9ef837368174eddb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:33.380726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312e736fc3e64a0c9ef837368174eddb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:33.387593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312e736fc3e64a0c9ef837368174eddb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:33.388228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312e736fc3e64a0c9ef837368174eddb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:33.406877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1050/2112] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:33 2022] DELETE /identity/v3/users/312e736fc3e64a0c9ef837368174eddb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:33.412130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ced546-66eb-4e5e-81d3-d8c0bd80205f None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:33.412598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ced546-66eb-4e5e-81d3-d8c0bd80205f None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:33.412692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ced546-66eb-4e5e-81d3-d8c0bd80205f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:33.431570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-a0ced546-66eb-4e5e-81d3-d8c0bd80205f None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:33.432783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a0ced546-66eb-4e5e-81d3-d8c0bd80205f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:33.494584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1063/2113] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:33.508979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:33.512976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:33.524074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:33.585470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c55695-c6e7-4ead-a092-ba4e4478069a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:33.589134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931f0f2-e269-4feb-afb2-17b462d024af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:33.589739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931f0f2-e269-4feb-afb2-17b462d024af service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:33.590020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931f0f2-e269-4feb-afb2-17b462d024af service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:33.590293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931f0f2-e269-4feb-afb2-17b462d024af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:33.599158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931f0f2-e269-4feb-afb2-17b462d024af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:33.600655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931f0f2-e269-4feb-afb2-17b462d024af service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:33.610271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1051/2114] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:33.889993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ced546-66eb-4e5e-81d3-d8c0bd80205f None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:33.891808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a0ced546-66eb-4e5e-81d3-d8c0bd80205f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:33.910520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ced546-66eb-4e5e-81d3-d8c0bd80205f None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:33.916892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:33.918053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:33.918600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:33.919085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] PATH_INFO: `/v3/projects/ef3f991ce37a447f8945326d323f5e20` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:33.920566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef3f991ce37a447f8945326d323f5e20)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:33.923254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:33.933819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:33.995294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef3f991ce37a447f8945326d323f5e20', 'request_id': 'req-439594b2-0711-47c0-ad93-69118f2b6f1e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.002657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef3f991ce37a447f8945326d323f5e20', 'request_id': 'req-439594b2-0711-47c0-ad93-69118f2b6f1e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.015445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef3f991ce37a447f8945326d323f5e20', 'request_id': 'req-439594b2-0711-47c0-ad93-69118f2b6f1e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.043524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] The token cache is being invalidate because project ef3f991ce37a447f8945326d323f5e20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:34.043843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.044584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.049784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1064/2115] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:33 2022] DELETE /identity/v3/projects/ef3f991ce37a447f8945326d323f5e20 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:34.051966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931f0f2-e269-4feb-afb2-17b462d024af service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:34.076304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931f0f2-e269-4feb-afb2-17b462d024af service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:34.144034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:34.144709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:34.145015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:34.145287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] PATH_INFO: `/v3/users/2cee7819eff9425f932f2216c799c2db` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:34.146411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cee7819eff9425f932f2216c799c2db)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:34.148363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:34.194980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cee7819eff9425f932f2216c799c2db'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.202573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cee7819eff9425f932f2216c799c2db'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.203201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cee7819eff9425f932f2216c799c2db'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.210329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cee7819eff9425f932f2216c799c2db'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.210977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-61a77b54-dc03-4f80-a9cb-17bce2eff1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cee7819eff9425f932f2216c799c2db'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.227858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1052/2116] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:34 2022] DELETE /identity/v3/users/2cee7819eff9425f932f2216c799c2db => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:34.508693 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:34.510738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:34.531067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439594b2-0711-47c0-ad93-69118f2b6f1e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:34.607475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:34.608877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:34.608877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:34.609329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] PATH_INFO: `/v3/projects/f035b78f4a824f349bacd85f8b90735c` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:34.610558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f035b78f4a824f349bacd85f8b90735c)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:34.612573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:34.669828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f035b78f4a824f349bacd85f8b90735c', 'request_id': 'req-5344a104-00ff-4b34-b1c5-5dbc83384623'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.676867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f035b78f4a824f349bacd85f8b90735c', 'request_id': 'req-5344a104-00ff-4b34-b1c5-5dbc83384623'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.685394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f035b78f4a824f349bacd85f8b90735c', 'request_id': 'req-5344a104-00ff-4b34-b1c5-5dbc83384623'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.700677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] The token cache is being invalidate because project f035b78f4a824f349bacd85f8b90735c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:34.701005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.701661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5344a104-00ff-4b34-b1c5-5dbc83384623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:34.703204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1065/2117] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:34 2022] DELETE /identity/v3/projects/f035b78f4a824f349bacd85f8b90735c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:34.708352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455db1e-a2a4-4e96-a5d9-02bb351a5fd0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:34.708863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455db1e-a2a4-4e96-a5d9-02bb351a5fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:34.709124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455db1e-a2a4-4e96-a5d9-02bb351a5fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:34.729066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9455db1e-a2a4-4e96-a5d9-02bb351a5fd0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:34.730293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9455db1e-a2a4-4e96-a5d9-02bb351a5fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:34.757218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:34.799257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1053/2118] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:34.807639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2828-5fea-49e9-84c9-1ab90f9b39c8 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:34.807989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2828-5fea-49e9-84c9-1ab90f9b39c8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:34.808346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2828-5fea-49e9-84c9-1ab90f9b39c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:34.839178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d54f2828-5fea-49e9-84c9-1ab90f9b39c8 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:34.840518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d54f2828-5fea-49e9-84c9-1ab90f9b39c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:34.853526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1066/2119] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:34.854601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455db1e-a2a4-4e96-a5d9-02bb351a5fd0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:34.867370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455db1e-a2a4-4e96-a5d9-02bb351a5fd0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:34.872833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789b5ce-beac-4dd1-82e6-0d58650c6e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:34.873576 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789b5ce-beac-4dd1-82e6-0d58650c6e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:34.873857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789b5ce-beac-4dd1-82e6-0d58650c6e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:34.874133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789b5ce-beac-4dd1-82e6-0d58650c6e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:34.874777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789b5ce-beac-4dd1-82e6-0d58650c6e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:34.875085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789b5ce-beac-4dd1-82e6-0d58650c6e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:34.876813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789b5ce-beac-4dd1-82e6-0d58650c6e6c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:34.889631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1054/2120] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:58:34.893742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2828-5fea-49e9-84c9-1ab90f9b39c8 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:34.903495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2828-5fea-49e9-84c9-1ab90f9b39c8 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:34.908273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f190cb-e400-4ee0-a563-3324c5ffd03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:34.908943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f190cb-e400-4ee0-a563-3324c5ffd03f admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:34.909226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f190cb-e400-4ee0-a563-3324c5ffd03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:34.909495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f190cb-e400-4ee0-a563-3324c5ffd03f admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:34.910768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f190cb-e400-4ee0-a563-3324c5ffd03f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:34.912903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f190cb-e400-4ee0-a563-3324c5ffd03f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:34.935060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:34.945193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1067/2121] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:34 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:34.949719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789b5ce-beac-4dd1-82e6-0d58650c6e6c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:34.959944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789b5ce-beac-4dd1-82e6-0d58650c6e6c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.019398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d3525-34a8-4f15-bb2d-039a632a5ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.020045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d3525-34a8-4f15-bb2d-039a632a5ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.020314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d3525-34a8-4f15-bb2d-039a632a5ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.020620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d3525-34a8-4f15-bb2d-039a632a5ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.021312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d3525-34a8-4f15-bb2d-039a632a5ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.023347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d3525-34a8-4f15-bb2d-039a632a5ca6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.048542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1055/2122] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:34 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:35.049846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f190cb-e400-4ee0-a563-3324c5ffd03f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.060618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f190cb-e400-4ee0-a563-3324c5ffd03f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.065377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55d7d5-0bc2-4ed7-9f8e-4dfe3df89a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.065981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55d7d5-0bc2-4ed7-9f8e-4dfe3df89a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.066353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55d7d5-0bc2-4ed7-9f8e-4dfe3df89a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.066633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55d7d5-0bc2-4ed7-9f8e-4dfe3df89a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.067781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55d7d5-0bc2-4ed7-9f8e-4dfe3df89a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.069442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55d7d5-0bc2-4ed7-9f8e-4dfe3df89a55 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.080095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1068/2123] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:35.084543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d3525-34a8-4f15-bb2d-039a632a5ca6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.094972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d3525-34a8-4f15-bb2d-039a632a5ca6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.099794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c8793-fd45-427c-9594-be017d316f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.100402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8793-fd45-427c-9594-be017d316f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.100831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8793-fd45-427c-9594-be017d316f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.101115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8793-fd45-427c-9594-be017d316f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.101755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c8793-fd45-427c-9594-be017d316f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.103500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c8793-fd45-427c-9594-be017d316f54 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.117322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1056/2124] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:35.123191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55d7d5-0bc2-4ed7-9f8e-4dfe3df89a55 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.135927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55d7d5-0bc2-4ed7-9f8e-4dfe3df89a55 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.141101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364dc3b-362c-4e14-b5d6-6c3416ff94e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.141881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364dc3b-362c-4e14-b5d6-6c3416ff94e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.142114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364dc3b-362c-4e14-b5d6-6c3416ff94e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.142392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364dc3b-362c-4e14-b5d6-6c3416ff94e5 admin admin] PATH_INFO: `/v3/projects/ecc2c6c064a44d99800f6ff8501da798/users/982a64f479014c8abbdfb5491654abed/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.160123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364dc3b-362c-4e14-b5d6-6c3416ff94e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc2c6c064a44d99800f6ff8501da798, user_id=982a64f479014c8abbdfb5491654abed, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.165259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364dc3b-362c-4e14-b5d6-6c3416ff94e5 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.192479 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1069/2125] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:35 2022] PUT /identity/v3/projects/ecc2c6c064a44d99800f6ff8501da798/users/982a64f479014c8abbdfb5491654abed/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:35.194749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d265e1e-c1db-48e9-9a93-3e27cf44c8cb None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.195100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d265e1e-c1db-48e9-9a93-3e27cf44c8cb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.195362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d265e1e-c1db-48e9-9a93-3e27cf44c8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.230393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0d265e1e-c1db-48e9-9a93-3e27cf44c8cb None None] MFA Rules not processed for user `982a64f479014c8abbdfb5491654abed`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:35.232131 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0d265e1e-c1db-48e9-9a93-3e27cf44c8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:35.314143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1057/2126] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:35.319473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364dc3b-362c-4e14-b5d6-6c3416ff94e5 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.385145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364dc3b-362c-4e14-b5d6-6c3416ff94e5 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.389789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9c5f4-a494-41f7-b21e-a4682ec91b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.390392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9c5f4-a494-41f7-b21e-a4682ec91b13 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.390673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9c5f4-a494-41f7-b21e-a4682ec91b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.390942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9c5f4-a494-41f7-b21e-a4682ec91b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.391580 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9c5f4-a494-41f7-b21e-a4682ec91b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.393542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9c5f4-a494-41f7-b21e-a4682ec91b13 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.419820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1070/2127] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:35 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:35.423647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d265e1e-c1db-48e9-9a93-3e27cf44c8cb None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.435002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d265e1e-c1db-48e9-9a93-3e27cf44c8cb None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.454681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:35.496083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564c5ee-ad8c-4b79-91ae-cfd508d4b67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.496905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564c5ee-ad8c-4b79-91ae-cfd508d4b67a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.497478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564c5ee-ad8c-4b79-91ae-cfd508d4b67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.497593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564c5ee-ad8c-4b79-91ae-cfd508d4b67a admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.498200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564c5ee-ad8c-4b79-91ae-cfd508d4b67a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.500540 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564c5ee-ad8c-4b79-91ae-cfd508d4b67a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.529210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1058/2128] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:35 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:35.533826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9c5f4-a494-41f7-b21e-a4682ec91b13 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.561190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9c5f4-a494-41f7-b21e-a4682ec91b13 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.566286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014b344-72d6-42e7-8cfb-2d9fbb331385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.567472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014b344-72d6-42e7-8cfb-2d9fbb331385 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.567472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014b344-72d6-42e7-8cfb-2d9fbb331385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.567702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014b344-72d6-42e7-8cfb-2d9fbb331385 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.568158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014b344-72d6-42e7-8cfb-2d9fbb331385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.570382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014b344-72d6-42e7-8cfb-2d9fbb331385 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.581558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1071/2129] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:35.586222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564c5ee-ad8c-4b79-91ae-cfd508d4b67a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.596872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564c5ee-ad8c-4b79-91ae-cfd508d4b67a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.602480 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aad7b7-1b6c-4c9a-8e4f-ab6950edd2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.603139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad7b7-1b6c-4c9a-8e4f-ab6950edd2a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.603418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad7b7-1b6c-4c9a-8e4f-ab6950edd2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.603685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad7b7-1b6c-4c9a-8e4f-ab6950edd2a8 admin admin] PATH_INFO: `/v3/projects/47bcd628370547679153eaf0b8c49477/users/5b727f844d6d4d60aa3a01122f6d004f/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.622250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aad7b7-1b6c-4c9a-8e4f-ab6950edd2a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bcd628370547679153eaf0b8c49477, user_id=5b727f844d6d4d60aa3a01122f6d004f, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.624953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aad7b7-1b6c-4c9a-8e4f-ab6950edd2a8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.639533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1059/2130] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:35 2022] PUT /identity/v3/projects/47bcd628370547679153eaf0b8c49477/users/5b727f844d6d4d60aa3a01122f6d004f/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:35.643481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014b344-72d6-42e7-8cfb-2d9fbb331385 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.653784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014b344-72d6-42e7-8cfb-2d9fbb331385 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.713271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfd7bd-806a-4058-a176-0e368f3fde4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.713911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd7bd-806a-4058-a176-0e368f3fde4a admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.714195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd7bd-806a-4058-a176-0e368f3fde4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.714467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd7bd-806a-4058-a176-0e368f3fde4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.715233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfd7bd-806a-4058-a176-0e368f3fde4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.716922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfd7bd-806a-4058-a176-0e368f3fde4a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.727530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1072/2131] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:35.731766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aad7b7-1b6c-4c9a-8e4f-ab6950edd2a8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.743150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aad7b7-1b6c-4c9a-8e4f-ab6950edd2a8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:35.747728 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c49a5-da2a-4aa4-b97a-6686b8008963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:35.748381 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c49a5-da2a-4aa4-b97a-6686b8008963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.748662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c49a5-da2a-4aa4-b97a-6686b8008963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.748970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c49a5-da2a-4aa4-b97a-6686b8008963 admin admin] PATH_INFO: `/v3/projects/47bcd628370547679153eaf0b8c49477/users/5b727f844d6d4d60aa3a01122f6d004f/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.750870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c49a5-da2a-4aa4-b97a-6686b8008963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bcd628370547679153eaf0b8c49477, user_id=5b727f844d6d4d60aa3a01122f6d004f, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:35.753599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c49a5-da2a-4aa4-b97a-6686b8008963 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:35.765147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1060/2132] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:35 2022] PUT /identity/v3/projects/47bcd628370547679153eaf0b8c49477/users/5b727f844d6d4d60aa3a01122f6d004f/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:35.776674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3c058-051c-4790-b8fd-f55ad4ef9fae None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:35.777232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3c058-051c-4790-b8fd-f55ad4ef9fae None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:35.777738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3c058-051c-4790-b8fd-f55ad4ef9fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:35.810111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-b4f3c058-051c-4790-b8fd-f55ad4ef9fae None None] MFA Rules not processed for user `5b727f844d6d4d60aa3a01122f6d004f`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:35.811347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b4f3c058-051c-4790-b8fd-f55ad4ef9fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:35.834556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:35.893085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1073/2133] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:35.913668 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c49a5-da2a-4aa4-b97a-6686b8008963 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:35.915637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1f9c49a5-da2a-4aa4-b97a-6686b8008963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:36.042395 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c49a5-da2a-4aa4-b97a-6686b8008963 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:36.050028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1bc01-45b6-4016-80ef-cdf35cb13437 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:36.050028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1bc01-45b6-4016-80ef-cdf35cb13437 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:36.050242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1bc01-45b6-4016-80ef-cdf35cb13437 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:36.050540 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1bc01-45b6-4016-80ef-cdf35cb13437 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:36.149825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1bc01-45b6-4016-80ef-cdf35cb13437 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:36.151820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1bc01-45b6-4016-80ef-cdf35cb13437 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:36.164393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1061/2134] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:40.561509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3c058-051c-4790-b8fd-f55ad4ef9fae None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:40.563031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b4f3c058-051c-4790-b8fd-f55ad4ef9fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:40.575478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3c058-051c-4790-b8fd-f55ad4ef9fae None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:40.579014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b5c2c-b491-4209-bd41-531ccdbc870a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:40.579617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b5c2c-b491-4209-bd41-531ccdbc870a service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:40.579914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b5c2c-b491-4209-bd41-531ccdbc870a service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:40.580165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b5c2c-b491-4209-bd41-531ccdbc870a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:40.589125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b5c2c-b491-4209-bd41-531ccdbc870a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:40.590560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b5c2c-b491-4209-bd41-531ccdbc870a service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:40.600379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1074/2135] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:42.991722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc4547-36a2-4701-b8df-04a01f94712b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:42.992422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc4547-36a2-4701-b8df-04a01f94712b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:42.992422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc4547-36a2-4701-b8df-04a01f94712b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:43.017086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0bbc4547-36a2-4701-b8df-04a01f94712b None None] MFA Rules not processed for user `c4139af75eec4ac6a1576f5a40b02e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:43.018293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0bbc4547-36a2-4701-b8df-04a01f94712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:43.044699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:43.076405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1062/2136] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:48.251141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcb24-ef96-4c05-8d3f-70b350bb37e0 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:48.251823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcb24-ef96-4c05-8d3f-70b350bb37e0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:48.251823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcb24-ef96-4c05-8d3f-70b350bb37e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:48.280312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-c07dcb24-ef96-4c05-8d3f-70b350bb37e0 None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:48.281825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c07dcb24-ef96-4c05-8d3f-70b350bb37e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:48.374244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1075/2137] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 10:58:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:48.427456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc4547-36a2-4701-b8df-04a01f94712b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:48.429147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0bbc4547-36a2-4701-b8df-04a01f94712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:48.547476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc4547-36a2-4701-b8df-04a01f94712b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:48.561940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261fd67-3424-4417-8601-f90b77f1ec0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:48.562579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261fd67-3424-4417-8601-f90b77f1ec0c service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:48.563337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261fd67-3424-4417-8601-f90b77f1ec0c service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:48.563337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261fd67-3424-4417-8601-f90b77f1ec0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:48.583390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261fd67-3424-4417-8601-f90b77f1ec0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:48.584945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261fd67-3424-4417-8601-f90b77f1ec0c service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:48.606977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1063/2138] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 10:58:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 10:58:49.224949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127c9c8-fc27-48da-a1fe-6e8f846aae83 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:49.225594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127c9c8-fc27-48da-a1fe-6e8f846aae83 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:49.225886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127c9c8-fc27-48da-a1fe-6e8f846aae83 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:49.230239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1076/2139] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 10:58:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:58:49.265110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48a100-fb60-4edb-906a-271cf9a2893c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:49.265495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48a100-fb60-4edb-906a-271cf9a2893c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:49.265739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48a100-fb60-4edb-906a-271cf9a2893c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:49.303508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7d48a100-fb60-4edb-906a-271cf9a2893c None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:49.305708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7d48a100-fb60-4edb-906a-271cf9a2893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:49.480959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1064/2140] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 10:58:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:49.503884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127c9c8-fc27-48da-a1fe-6e8f846aae83 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:49.506342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7127c9c8-fc27-48da-a1fe-6e8f846aae83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:49.557494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127c9c8-fc27-48da-a1fe-6e8f846aae83 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:49.570571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d998ea-408a-4f79-beec-0098b6e3db86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:49.574017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d998ea-408a-4f79-beec-0098b6e3db86 service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:49.574670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d998ea-408a-4f79-beec-0098b6e3db86 service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:49.575142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d998ea-408a-4f79-beec-0098b6e3db86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:49.588559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d998ea-408a-4f79-beec-0098b6e3db86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:49.591405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d998ea-408a-4f79-beec-0098b6e3db86 service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:49.610672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1077/2141] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:50.296307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48a100-fb60-4edb-906a-271cf9a2893c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:50.298267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7d48a100-fb60-4edb-906a-271cf9a2893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:50.378181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48a100-fb60-4edb-906a-271cf9a2893c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:50.383812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:50.384746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:50.384966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:50.385304 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] PATH_INFO: `/v3/users/baf3f8f4e8f740c5b9693e92fd074b3f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:50.386769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf3f8f4e8f740c5b9693e92fd074b3f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:50.389147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:50.392308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:50.455113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf3f8f4e8f740c5b9693e92fd074b3f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:50.474980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf3f8f4e8f740c5b9693e92fd074b3f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:50.476135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf3f8f4e8f740c5b9693e92fd074b3f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:50.485484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf3f8f4e8f740c5b9693e92fd074b3f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:50.486931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf3f8f4e8f740c5b9693e92fd074b3f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:50.506998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1065/2142] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:50 2022] DELETE /identity/v3/users/baf3f8f4e8f740c5b9693e92fd074b3f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:50.798257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d998ea-408a-4f79-beec-0098b6e3db86 service placement] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:50.847545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d998ea-408a-4f79-beec-0098b6e3db86 service placement] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:50.919525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:50.920211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:50.920488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:50.920769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] PATH_INFO: `/v3/projects/2c853fd25a4241998f2d6c6af65df780` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:50.922114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c853fd25a4241998f2d6c6af65df780)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:50.924074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:50.972959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:50.980774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c853fd25a4241998f2d6c6af65df780', 'request_id': 'req-58a51283-2fd1-4698-a6e2-89a2b4264926'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:50.987926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c853fd25a4241998f2d6c6af65df780', 'request_id': 'req-58a51283-2fd1-4698-a6e2-89a2b4264926'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:50.997030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c853fd25a4241998f2d6c6af65df780', 'request_id': 'req-58a51283-2fd1-4698-a6e2-89a2b4264926'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.011673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] The token cache is being invalidate because project 2c853fd25a4241998f2d6c6af65df780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:51.012015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.012642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.014496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1078/2143] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:50 2022] DELETE /identity/v3/projects/2c853fd25a4241998f2d6c6af65df780 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:51.018288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:51.019852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:51.034770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f98f0-fecc-4f24-b34f-986c7376eff9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:51.036167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:51.037686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:51.116090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:51.116764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:51.117051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:51.117325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] PATH_INFO: `/v3/users/6907bc47b2e44dac8fa3253ae98aad8e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:51.118534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6907bc47b2e44dac8fa3253ae98aad8e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:51.120479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:51.141518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a51283-2fd1-4698-a6e2-89a2b4264926 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:51.149638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:51.150525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:51.150815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:51.151078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] PATH_INFO: `/v3/users/982a64f479014c8abbdfb5491654abed` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:51.152338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] RBAC: Authorizing `identity:delete_user(user_id=982a64f479014c8abbdfb5491654abed)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:51.155362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:51.198995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6907bc47b2e44dac8fa3253ae98aad8e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.211865 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6907bc47b2e44dac8fa3253ae98aad8e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.212713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6907bc47b2e44dac8fa3253ae98aad8e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.226232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982a64f479014c8abbdfb5491654abed'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.227359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982a64f479014c8abbdfb5491654abed'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.227928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6907bc47b2e44dac8fa3253ae98aad8e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.228791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49927357-d9c8-495c-b395-af04b012fd0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6907bc47b2e44dac8fa3253ae98aad8e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.239087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982a64f479014c8abbdfb5491654abed'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.240133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982a64f479014c8abbdfb5491654abed'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.248368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:51.258587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1066/2144] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:51 2022] DELETE /identity/v3/users/6907bc47b2e44dac8fa3253ae98aad8e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:51.260715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982a64f479014c8abbdfb5491654abed'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.303839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1079/2145] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:51 2022] DELETE /identity/v3/users/982a64f479014c8abbdfb5491654abed => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:51.305820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c0f0-b4a5-4892-8872-c6bf0e867049 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:51.306499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c0f0-b4a5-4892-8872-c6bf0e867049 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:51.306846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c0f0-b4a5-4892-8872-c6bf0e867049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:51.329812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4b74c0f0-b4a5-4892-8872-c6bf0e867049 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:51.332014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4b74c0f0-b4a5-4892-8872-c6bf0e867049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:51.404132 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1067/2146] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:51.415994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:51.417609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:51.521801 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:51.545903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c0f0-b4a5-4892-8872-c6bf0e867049 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:51.548274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4b74c0f0-b4a5-4892-8872-c6bf0e867049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:51.555153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81e954-bad6-4ffb-abc6-bb275408072b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:51.567248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00cf2f-0501-41c8-9415-76940eba3066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:51.569807 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00cf2f-0501-41c8-9415-76940eba3066 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:51.572876 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00cf2f-0501-41c8-9415-76940eba3066 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:51.573262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00cf2f-0501-41c8-9415-76940eba3066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:51.597286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c0f0-b4a5-4892-8872-c6bf0e867049 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:51.604386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:51.605025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:51.605307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:51.605628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] PATH_INFO: `/v3/projects/8cdc953fcc854694a04ab4da5c366fb2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:51.605710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00cf2f-0501-41c8-9415-76940eba3066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:51.606798 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cdc953fcc854694a04ab4da5c366fb2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:51.609155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:51.610188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00cf2f-0501-41c8-9415-76940eba3066 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:51.636228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1080/2147] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:51.712098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cdc953fcc854694a04ab4da5c366fb2', 'request_id': 'req-d266c064-3946-4baf-a7c1-67f27334228d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.724599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cdc953fcc854694a04ab4da5c366fb2', 'request_id': 'req-d266c064-3946-4baf-a7c1-67f27334228d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.736150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cdc953fcc854694a04ab4da5c366fb2', 'request_id': 'req-d266c064-3946-4baf-a7c1-67f27334228d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.750311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] The token cache is being invalidate because project 8cdc953fcc854694a04ab4da5c366fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:51.750922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.751836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d266c064-3946-4baf-a7c1-67f27334228d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:51.754981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1068/2148] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:51 2022] DELETE /identity/v3/projects/8cdc953fcc854694a04ab4da5c366fb2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:51.759640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce4b60-92dd-4f68-bcc0-d7ade59a6fd8 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:51.759923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce4b60-92dd-4f68-bcc0-d7ade59a6fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:51.760244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce4b60-92dd-4f68-bcc0-d7ade59a6fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:51.789996 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-45ce4b60-92dd-4f68-bcc0-d7ade59a6fd8 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:51.792158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-45ce4b60-92dd-4f68-bcc0-d7ade59a6fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:51.899463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1081/2149] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:51.903080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6eb20-7e79-4a37-bf45-b584c45509a2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:51.903669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6eb20-7e79-4a37-bf45-b584c45509a2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:51.904066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6eb20-7e79-4a37-bf45-b584c45509a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:51.954898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e3e6eb20-7e79-4a37-bf45-b584c45509a2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:51.957157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e3e6eb20-7e79-4a37-bf45-b584c45509a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:51.966717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1069/2150] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:51.972573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce4b60-92dd-4f68-bcc0-d7ade59a6fd8 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:51.990892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce4b60-92dd-4f68-bcc0-d7ade59a6fd8 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:51.999885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462c8e5-aaa9-4325-89ae-797e65f9b22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:52.001609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462c8e5-aaa9-4325-89ae-797e65f9b22f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.001907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462c8e5-aaa9-4325-89ae-797e65f9b22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.002201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462c8e5-aaa9-4325-89ae-797e65f9b22f admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.004540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462c8e5-aaa9-4325-89ae-797e65f9b22f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:52.005439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462c8e5-aaa9-4325-89ae-797e65f9b22f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:52.007800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462c8e5-aaa9-4325-89ae-797e65f9b22f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:52.033617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1082/2151] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:58:52.038086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6eb20-7e79-4a37-bf45-b584c45509a2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:52.050067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6eb20-7e79-4a37-bf45-b584c45509a2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:52.057766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf857c-9f31-4d67-9d92-4fea68a0ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:52.058529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf857c-9f31-4d67-9d92-4fea68a0ea18 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.058848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf857c-9f31-4d67-9d92-4fea68a0ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.059115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf857c-9f31-4d67-9d92-4fea68a0ea18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.059780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf857c-9f31-4d67-9d92-4fea68a0ea18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:52.062136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf857c-9f31-4d67-9d92-4fea68a0ea18 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:52.106429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1070/2152] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:52 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:52.110442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462c8e5-aaa9-4325-89ae-797e65f9b22f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:52.134551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462c8e5-aaa9-4325-89ae-797e65f9b22f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:52.165305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf857c-9f31-4d67-9d92-4fea68a0ea18 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:52.167154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-51bf857c-9f31-4d67-9d92-4fea68a0ea18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:52.278116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fdadf-6d8b-4904-a114-a69ef5bab6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:52.278668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fdadf-6d8b-4904-a114-a69ef5bab6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.278951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fdadf-6d8b-4904-a114-a69ef5bab6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.279249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fdadf-6d8b-4904-a114-a69ef5bab6a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.279900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fdadf-6d8b-4904-a114-a69ef5bab6a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:52.282092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fdadf-6d8b-4904-a114-a69ef5bab6a4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:52.283371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:52.293547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf857c-9f31-4d67-9d92-4fea68a0ea18 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:52.304498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:52.305726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.306542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.307066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] PATH_INFO: `/v3/projects/ecc2c6c064a44d99800f6ff8501da798` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.309092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc2c6c064a44d99800f6ff8501da798)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:52.311661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1083/2153] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:52 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:52.312641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:52.315547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fdadf-6d8b-4904-a114-a69ef5bab6a4 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:52.326766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fdadf-6d8b-4904-a114-a69ef5bab6a4 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:52.333009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ecd7b-349c-49d6-a8d5-23b7a4ed906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:52.333779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ecd7b-349c-49d6-a8d5-23b7a4ed906c admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.334109 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ecd7b-349c-49d6-a8d5-23b7a4ed906c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.334383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ecd7b-349c-49d6-a8d5-23b7a4ed906c admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.335077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ecd7b-349c-49d6-a8d5-23b7a4ed906c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:52.336844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ecd7b-349c-49d6-a8d5-23b7a4ed906c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:52.360271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1084/2154] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:52.361365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ecd7b-349c-49d6-a8d5-23b7a4ed906c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:52.373398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ecd7b-349c-49d6-a8d5-23b7a4ed906c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:52.380123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c95be-9010-4bd1-a843-e1444b248671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:52.380770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c95be-9010-4bd1-a843-e1444b248671 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.381113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c95be-9010-4bd1-a843-e1444b248671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.381455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c95be-9010-4bd1-a843-e1444b248671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.382035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c95be-9010-4bd1-a843-e1444b248671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:52.383715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c95be-9010-4bd1-a843-e1444b248671 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:52.400068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1085/2155] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:58:52.401855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c95be-9010-4bd1-a843-e1444b248671 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:52.406883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc2c6c064a44d99800f6ff8501da798', 'request_id': 'req-16c40a50-d67b-4420-8520-c9e53441fbfb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.414386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c95be-9010-4bd1-a843-e1444b248671 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:52.416986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc2c6c064a44d99800f6ff8501da798', 'request_id': 'req-16c40a50-d67b-4420-8520-c9e53441fbfb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.428514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc2c6c064a44d99800f6ff8501da798', 'request_id': 'req-16c40a50-d67b-4420-8520-c9e53441fbfb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.447839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] The token cache is being invalidate because project ecc2c6c064a44d99800f6ff8501da798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:52.448127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.448795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.450341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1071/2156] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:52 2022] DELETE /identity/v3/projects/ecc2c6c064a44d99800f6ff8501da798 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:52.455631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:52.458957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:52.474695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:52.498655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a00d71-d15e-4695-ab17-4ef192fe2255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:52.499326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a00d71-d15e-4695-ab17-4ef192fe2255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.499607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a00d71-d15e-4695-ab17-4ef192fe2255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.499887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a00d71-d15e-4695-ab17-4ef192fe2255 admin admin] PATH_INFO: `/v3/projects/2cc0d847286b4b649839b3fdf8504fb2/users/7d72c79d7b4649cf8ddb70cd32eb088b/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.518400 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a00d71-d15e-4695-ab17-4ef192fe2255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc0d847286b4b649839b3fdf8504fb2, user_id=7d72c79d7b4649cf8ddb70cd32eb088b, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:52.522196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a00d71-d15e-4695-ab17-4ef192fe2255 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:52.552919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1086/2157] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:52 2022] PUT /identity/v3/projects/2cc0d847286b4b649839b3fdf8504fb2/users/7d72c79d7b4649cf8ddb70cd32eb088b/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:52.563078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c40a50-d67b-4420-8520-c9e53441fbfb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:52.565148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe996090-291f-458c-b1fb-9525f2ee12a2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.567103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe996090-291f-458c-b1fb-9525f2ee12a2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.567410 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe996090-291f-458c-b1fb-9525f2ee12a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.608106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-fe996090-291f-458c-b1fb-9525f2ee12a2 None None] MFA Rules not processed for user `7d72c79d7b4649cf8ddb70cd32eb088b`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:52.609916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fe996090-291f-458c-b1fb-9525f2ee12a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:52.664756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:52.665466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.665738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.666016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] PATH_INFO: `/v3/users/5b727f844d6d4d60aa3a01122f6d004f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.667201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b727f844d6d4d60aa3a01122f6d004f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:52.669559 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:52.719200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:52.736887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b727f844d6d4d60aa3a01122f6d004f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.751239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b727f844d6d4d60aa3a01122f6d004f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.754608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b727f844d6d4d60aa3a01122f6d004f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.759612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1087/2158] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:52.775073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b727f844d6d4d60aa3a01122f6d004f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.775316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe996090-291f-458c-b1fb-9525f2ee12a2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:52.775830 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b727f844d6d4d60aa3a01122f6d004f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:52.778673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fe996090-291f-458c-b1fb-9525f2ee12a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:52.799249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1072/2159] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:52 2022] DELETE /identity/v3/users/5b727f844d6d4d60aa3a01122f6d004f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:52.872415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe996090-291f-458c-b1fb-9525f2ee12a2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:52.938617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:52.941569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:52.942288 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:52.942942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:52.945248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:52.998787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:53.047917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:53.050111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:53.064932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1088/2160] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:53.100943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:53.102871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:53.240819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:53.243166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:53.255153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806e6b-6889-42e9-993f-0f04ee80ad63 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:53.261707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:53.262536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:53.262928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:53.263412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] PATH_INFO: `/v3/users/7d72c79d7b4649cf8ddb70cd32eb088b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:53.264962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d72c79d7b4649cf8ddb70cd32eb088b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:53.265285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544679f4-82bf-453e-a02f-fe02709ea284 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:53.268002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:53.273101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:53.274813 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:53.275298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:53.275802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] PATH_INFO: `/v3/projects/47bcd628370547679153eaf0b8c49477` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:53.285735 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] RBAC: Authorizing `identity:delete_project(project_id=47bcd628370547679153eaf0b8c49477)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:53.290707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:53.356729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d72c79d7b4649cf8ddb70cd32eb088b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.367651 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d72c79d7b4649cf8ddb70cd32eb088b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.368351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d72c79d7b4649cf8ddb70cd32eb088b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.378306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d72c79d7b4649cf8ddb70cd32eb088b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.379294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2748e2b9-2377-4f76-ac4a-64eb81d7805d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d72c79d7b4649cf8ddb70cd32eb088b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.426212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1073/2161] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:58:53 2022] DELETE /identity/v3/users/7d72c79d7b4649cf8ddb70cd32eb088b => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:53.428285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b0c96-3a7b-4afc-b719-25db23af4cdf None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:53.428419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47bcd628370547679153eaf0b8c49477', 'request_id': 'req-ece50821-a51f-4c67-8829-0cefa5eee53f'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.428699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b0c96-3a7b-4afc-b719-25db23af4cdf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:53.429089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b0c96-3a7b-4afc-b719-25db23af4cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:53.443732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47bcd628370547679153eaf0b8c49477', 'request_id': 'req-ece50821-a51f-4c67-8829-0cefa5eee53f'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.457451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e81b0c96-3a7b-4afc-b719-25db23af4cdf None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:53.458813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e81b0c96-3a7b-4afc-b719-25db23af4cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:53.470678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47bcd628370547679153eaf0b8c49477', 'request_id': 'req-ece50821-a51f-4c67-8829-0cefa5eee53f'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.493903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] The token cache is being invalidate because project 47bcd628370547679153eaf0b8c49477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:53.494578 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.495513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ece50821-a51f-4c67-8829-0cefa5eee53f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:53.498440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1089/2162] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:53 2022] DELETE /identity/v3/projects/47bcd628370547679153eaf0b8c49477 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:53.501058 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:53.506806 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb47e5-9c41-436c-b020-3bea7f2f8b7e None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:53.507393 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb47e5-9c41-436c-b020-3bea7f2f8b7e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:53.508160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb47e5-9c41-436c-b020-3bea7f2f8b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:53.546453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-19fb47e5-9c41-436c-b020-3bea7f2f8b7e None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:53.547794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-19fb47e5-9c41-436c-b020-3bea7f2f8b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:53.592229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1074/2163] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:53.603578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b0c96-3a7b-4afc-b719-25db23af4cdf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:53.606789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e81b0c96-3a7b-4afc-b719-25db23af4cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:53.666858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1090/2164] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:53.677868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f5c3f-153a-41f5-911e-13608fc3794e None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:53.678113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f5c3f-153a-41f5-911e-13608fc3794e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:53.678371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f5c3f-153a-41f5-911e-13608fc3794e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:53.720904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b0c96-3a7b-4afc-b719-25db23af4cdf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:53.730321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed7e7a-ccbe-45cc-b6af-41ccbf699b34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:53.731192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed7e7a-ccbe-45cc-b6af-41ccbf699b34 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:53.731916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5e6f5c3f-153a-41f5-911e-13608fc3794e None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:53.732196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed7e7a-ccbe-45cc-b6af-41ccbf699b34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:53.732572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed7e7a-ccbe-45cc-b6af-41ccbf699b34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:53.734151 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5e6f5c3f-153a-41f5-911e-13608fc3794e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:53.746119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed7e7a-ccbe-45cc-b6af-41ccbf699b34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:53.747930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed7e7a-ccbe-45cc-b6af-41ccbf699b34 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:53.766859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1075/2165] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:53.777575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1091/2166] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:53.783512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed7e7a-ccbe-45cc-b6af-41ccbf699b34 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:53.825314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed7e7a-ccbe-45cc-b6af-41ccbf699b34 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:53.837174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec72c4-0326-4344-804d-3655a18831d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:53.838043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec72c4-0326-4344-804d-3655a18831d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:53.838435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec72c4-0326-4344-804d-3655a18831d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:53.838928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec72c4-0326-4344-804d-3655a18831d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:53.839613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec72c4-0326-4344-804d-3655a18831d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:53.839928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec72c4-0326-4344-804d-3655a18831d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:53.841759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec72c4-0326-4344-804d-3655a18831d9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:53.860946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1076/2167] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:58:53.863183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f5c3f-153a-41f5-911e-13608fc3794e None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:53.881461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f5c3f-153a-41f5-911e-13608fc3794e None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:53.890327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e68d5-2710-4a8b-95fe-cc08d750f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:53.891249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e68d5-2710-4a8b-95fe-cc08d750f3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:53.891848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e68d5-2710-4a8b-95fe-cc08d750f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:53.892473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e68d5-2710-4a8b-95fe-cc08d750f3e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:53.894271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e68d5-2710-4a8b-95fe-cc08d750f3e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:53.896950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e68d5-2710-4a8b-95fe-cc08d750f3e3 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:53.955394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1092/2168] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:53 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:53.963060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec72c4-0326-4344-804d-3655a18831d9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:53.997207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec72c4-0326-4344-804d-3655a18831d9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:54.150888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb2afe-41d3-4583-9c68-764bce6ad45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:54.151903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb2afe-41d3-4583-9c68-764bce6ad45a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.152547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb2afe-41d3-4583-9c68-764bce6ad45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.153064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb2afe-41d3-4583-9c68-764bce6ad45a admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.155284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb2afe-41d3-4583-9c68-764bce6ad45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:54.157527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb2afe-41d3-4583-9c68-764bce6ad45a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:54.207274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1077/2169] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:53 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:54.211769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e68d5-2710-4a8b-95fe-cc08d750f3e3 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:54.223848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e68d5-2710-4a8b-95fe-cc08d750f3e3 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:54.235095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835cebd-c17e-4a79-ac5e-85a886e78136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:54.238045 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835cebd-c17e-4a79-ac5e-85a886e78136 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.239165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835cebd-c17e-4a79-ac5e-85a886e78136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.239625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835cebd-c17e-4a79-ac5e-85a886e78136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.242437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835cebd-c17e-4a79-ac5e-85a886e78136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:54.246493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835cebd-c17e-4a79-ac5e-85a886e78136 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:54.270115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1093/2170] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:54.274543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb2afe-41d3-4583-9c68-764bce6ad45a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:54.288422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb2afe-41d3-4583-9c68-764bce6ad45a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:54.300542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c4c91-9ed3-4d1f-a1f0-b845a04f63e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:54.301490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c4c91-9ed3-4d1f-a1f0-b845a04f63e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.301969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c4c91-9ed3-4d1f-a1f0-b845a04f63e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.302438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c4c91-9ed3-4d1f-a1f0-b845a04f63e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.303297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c4c91-9ed3-4d1f-a1f0-b845a04f63e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:54.306567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c4c91-9ed3-4d1f-a1f0-b845a04f63e8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:54.321248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1078/2171] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:54.326172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835cebd-c17e-4a79-ac5e-85a886e78136 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:54.338288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c4c91-9ed3-4d1f-a1f0-b845a04f63e8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:54.340868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8d0c4c91-9ed3-4d1f-a1f0-b845a04f63e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:54.345280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835cebd-c17e-4a79-ac5e-85a886e78136 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:54.355220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c3898-84d7-47a9-821f-2cf61be92f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:54.356154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c3898-84d7-47a9-821f-2cf61be92f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.356884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c3898-84d7-47a9-821f-2cf61be92f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.357405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c3898-84d7-47a9-821f-2cf61be92f1c admin admin] PATH_INFO: `/v3/projects/c699c5bed83549a0be701b20ead4023c/users/b3768ea847d34de188db083379b0e1e5/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.375783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:54.390672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c4c91-9ed3-4d1f-a1f0-b845a04f63e8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:54.391875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c3898-84d7-47a9-821f-2cf61be92f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c699c5bed83549a0be701b20ead4023c, user_id=b3768ea847d34de188db083379b0e1e5, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:54.396532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c3898-84d7-47a9-821f-2cf61be92f1c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:54.397418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:54.398340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.399049 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.399808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] PATH_INFO: `/v3/projects/2cc0d847286b4b649839b3fdf8504fb2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.405670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc0d847286b4b649839b3fdf8504fb2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:54.421222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:54.427444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1094/2172] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:54 2022] PUT /identity/v3/projects/c699c5bed83549a0be701b20ead4023c/users/b3768ea847d34de188db083379b0e1e5/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:54.442437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2aaf6-62b1-4f0b-8ffe-5c30ad0d1cd2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.442812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2aaf6-62b1-4f0b-8ffe-5c30ad0d1cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.443022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2aaf6-62b1-4f0b-8ffe-5c30ad0d1cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.486723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-48b2aaf6-62b1-4f0b-8ffe-5c30ad0d1cd2 None None] MFA Rules not processed for user `b3768ea847d34de188db083379b0e1e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:54.490141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-48b2aaf6-62b1-4f0b-8ffe-5c30ad0d1cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:54.498598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:54.515916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc0d847286b4b649839b3fdf8504fb2', 'request_id': 'req-346d93cf-ed29-4fa6-9301-711b9d0d33a3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:54.524214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc0d847286b4b649839b3fdf8504fb2', 'request_id': 'req-346d93cf-ed29-4fa6-9301-711b9d0d33a3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:54.535894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc0d847286b4b649839b3fdf8504fb2', 'request_id': 'req-346d93cf-ed29-4fa6-9301-711b9d0d33a3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:54.562363 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] The token cache is being invalidate because project 2cc0d847286b4b649839b3fdf8504fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:58:54.562852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:54.563676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-346d93cf-ed29-4fa6-9301-711b9d0d33a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:58:54.570180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1079/2173] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:58:54 2022] DELETE /identity/v3/projects/2cc0d847286b4b649839b3fdf8504fb2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:54.571575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9444aa-9893-4f14-a71b-ddbbe9d0b1f4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.571855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9444aa-9893-4f14-a71b-ddbbe9d0b1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.572449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9444aa-9893-4f14-a71b-ddbbe9d0b1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.594630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4c9444aa-9893-4f14-a71b-ddbbe9d0b1f4 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:54.595833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4c9444aa-9893-4f14-a71b-ddbbe9d0b1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:54.611095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1095/2174] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:54.618456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2aaf6-62b1-4f0b-8ffe-5c30ad0d1cd2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:54.622064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-48b2aaf6-62b1-4f0b-8ffe-5c30ad0d1cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:54.700935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1080/2175] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:54.708563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f56f8-fa73-40b8-89a2-bd4529170494 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.708938 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f56f8-fa73-40b8-89a2-bd4529170494 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.709147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f56f8-fa73-40b8-89a2-bd4529170494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.733724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-307f56f8-fa73-40b8-89a2-bd4529170494 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:54.735813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-307f56f8-fa73-40b8-89a2-bd4529170494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:54.747770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2aaf6-62b1-4f0b-8ffe-5c30ad0d1cd2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:54.749496 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1081/2176] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:54.754744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f56f8-fa73-40b8-89a2-bd4529170494 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:54.759333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca79250-453b-4a72-8e60-58aed1140520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:54.760334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca79250-453b-4a72-8e60-58aed1140520 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.760619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca79250-453b-4a72-8e60-58aed1140520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.760924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca79250-453b-4a72-8e60-58aed1140520 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.761572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca79250-453b-4a72-8e60-58aed1140520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:54.764785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca79250-453b-4a72-8e60-58aed1140520 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:54.789995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f56f8-fa73-40b8-89a2-bd4529170494 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:54.799122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b950b-b436-4d09-83a8-b19b5b3792ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:54.800591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b950b-b436-4d09-83a8-b19b5b3792ae admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.802375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b950b-b436-4d09-83a8-b19b5b3792ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.802375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b950b-b436-4d09-83a8-b19b5b3792ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.802785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b950b-b436-4d09-83a8-b19b5b3792ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:58:54.803477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b950b-b436-4d09-83a8-b19b5b3792ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:54.806803 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b950b-b436-4d09-83a8-b19b5b3792ae admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:54.830265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1096/2177] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:54 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:54.830502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1082/2178] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:58:54.833973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b950b-b436-4d09-83a8-b19b5b3792ae admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:54.836459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca79250-453b-4a72-8e60-58aed1140520 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:54.846052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b950b-b436-4d09-83a8-b19b5b3792ae admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:54.960050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca79250-453b-4a72-8e60-58aed1140520 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:54.962413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebb8e3-ea28-4f28-be36-da85dcfb0410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:54.963035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebb8e3-ea28-4f28-be36-da85dcfb0410 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.963322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebb8e3-ea28-4f28-be36-da85dcfb0410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.963626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebb8e3-ea28-4f28-be36-da85dcfb0410 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.964367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebb8e3-ea28-4f28-be36-da85dcfb0410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:54.966431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebb8e3-ea28-4f28-be36-da85dcfb0410 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:54.967093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84271f-3ed5-4bd2-a16c-04f94f614ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:54.967890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84271f-3ed5-4bd2-a16c-04f94f614ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:54.968479 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84271f-3ed5-4bd2-a16c-04f94f614ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:54.968760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84271f-3ed5-4bd2-a16c-04f94f614ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:54.969467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84271f-3ed5-4bd2-a16c-04f94f614ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:54.972044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84271f-3ed5-4bd2-a16c-04f94f614ec7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:54.980777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:54.997037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1083/2179] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:54 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:58:54.999731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebb8e3-ea28-4f28-be36-da85dcfb0410 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.013896 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebb8e3-ea28-4f28-be36-da85dcfb0410 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.021202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1097/2180] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:54 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:55.021789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28061c-f86c-483b-aec2-5c9ab969e0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.022148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84271f-3ed5-4bd2-a16c-04f94f614ec7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.022529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28061c-f86c-483b-aec2-5c9ab969e0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.022714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28061c-f86c-483b-aec2-5c9ab969e0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.023005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28061c-f86c-483b-aec2-5c9ab969e0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.023742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28061c-f86c-483b-aec2-5c9ab969e0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.025806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28061c-f86c-483b-aec2-5c9ab969e0d1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.041800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1084/2181] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:55.043518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28061c-f86c-483b-aec2-5c9ab969e0d1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.057273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28061c-f86c-483b-aec2-5c9ab969e0d1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.083612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:55.125729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84271f-3ed5-4bd2-a16c-04f94f614ec7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.132892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015755d0-0604-4566-ab45-bff816fa9392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.133533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015755d0-0604-4566-ab45-bff816fa9392 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.134065 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015755d0-0604-4566-ab45-bff816fa9392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.134065 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015755d0-0604-4566-ab45-bff816fa9392 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.135406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015755d0-0604-4566-ab45-bff816fa9392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.138573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015755d0-0604-4566-ab45-bff816fa9392 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.157393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d811bdd-dd96-4a23-a3b6-46c8ccca6300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.158460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d811bdd-dd96-4a23-a3b6-46c8ccca6300 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.159716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d811bdd-dd96-4a23-a3b6-46c8ccca6300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.160327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d811bdd-dd96-4a23-a3b6-46c8ccca6300 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.161174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d811bdd-dd96-4a23-a3b6-46c8ccca6300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.163858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d811bdd-dd96-4a23-a3b6-46c8ccca6300 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.168110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1098/2182] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:55 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:55.183261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015755d0-0604-4566-ab45-bff816fa9392 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.193201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1085/2183] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:55.196510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015755d0-0604-4566-ab45-bff816fa9392 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.197824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d811bdd-dd96-4a23-a3b6-46c8ccca6300 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.202424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74368564-67d9-4a59-9528-7bdac3b9b4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.203118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74368564-67d9-4a59-9528-7bdac3b9b4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.203693 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74368564-67d9-4a59-9528-7bdac3b9b4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.204059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74368564-67d9-4a59-9528-7bdac3b9b4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.204819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74368564-67d9-4a59-9528-7bdac3b9b4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.207762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74368564-67d9-4a59-9528-7bdac3b9b4d1 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.212659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d811bdd-dd96-4a23-a3b6-46c8ccca6300 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.218735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48db857-6e8d-41f5-86e4-5b49dd3c2eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.219523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48db857-6e8d-41f5-86e4-5b49dd3c2eac admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.219883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48db857-6e8d-41f5-86e4-5b49dd3c2eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.220245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48db857-6e8d-41f5-86e4-5b49dd3c2eac admin admin] PATH_INFO: `/v3/projects/fa37a295342042dca113e316e643af20/users/ec4e9cb7826d47f5908a81dbfea7b02a/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.225451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1099/2184] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:55.229232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74368564-67d9-4a59-9528-7bdac3b9b4d1 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.240706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74368564-67d9-4a59-9528-7bdac3b9b4d1 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.241861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48db857-6e8d-41f5-86e4-5b49dd3c2eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa37a295342042dca113e316e643af20, user_id=ec4e9cb7826d47f5908a81dbfea7b02a, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.244698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48db857-6e8d-41f5-86e4-5b49dd3c2eac admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.245979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8087f-b639-4003-931d-28cd03b3d893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.246577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8087f-b639-4003-931d-28cd03b3d893 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.246857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8087f-b639-4003-931d-28cd03b3d893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.247322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8087f-b639-4003-931d-28cd03b3d893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.247964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8087f-b639-4003-931d-28cd03b3d893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.251063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8087f-b639-4003-931d-28cd03b3d893 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.260539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1086/2185] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:55 2022] PUT /identity/v3/projects/fa37a295342042dca113e316e643af20/users/ec4e9cb7826d47f5908a81dbfea7b02a/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:58:55.265625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1100/2186] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:55.266908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48db857-6e8d-41f5-86e4-5b49dd3c2eac admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.273069 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6acb48-df0f-4ef2-89ea-54bb097c96c3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.273575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6acb48-df0f-4ef2-89ea-54bb097c96c3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.273846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6acb48-df0f-4ef2-89ea-54bb097c96c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.312521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3b6acb48-df0f-4ef2-89ea-54bb097c96c3 None None] MFA Rules not processed for user `ec4e9cb7826d47f5908a81dbfea7b02a`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:55.314756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3b6acb48-df0f-4ef2-89ea-54bb097c96c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:55.345347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48db857-6e8d-41f5-86e4-5b49dd3c2eac admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.353321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376fada-eb03-43fd-825d-e6dc8187883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.354299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376fada-eb03-43fd-825d-e6dc8187883a admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.354762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376fada-eb03-43fd-825d-e6dc8187883a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.355206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376fada-eb03-43fd-825d-e6dc8187883a admin admin] PATH_INFO: `/v3/projects/be8add5c995e456f9323d597079c1623/users/c0da8726d48b42f0bc70ebcd4b32c887/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.374912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376fada-eb03-43fd-825d-e6dc8187883a admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8add5c995e456f9323d597079c1623, user_id=c0da8726d48b42f0bc70ebcd4b32c887, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.378029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376fada-eb03-43fd-825d-e6dc8187883a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.395088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1087/2187] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:55 2022] PUT /identity/v3/projects/be8add5c995e456f9323d597079c1623/users/c0da8726d48b42f0bc70ebcd4b32c887/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:55.414488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1101/2188] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:55.419815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376fada-eb03-43fd-825d-e6dc8187883a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.475364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275552d-e3e0-4cb8-9a96-82e77d66c0ce None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.475892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275552d-e3e0-4cb8-9a96-82e77d66c0ce None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.476168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275552d-e3e0-4cb8-9a96-82e77d66c0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.513586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-6275552d-e3e0-4cb8-9a96-82e77d66c0ce None None] MFA Rules not processed for user `c0da8726d48b42f0bc70ebcd4b32c887`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:55.515259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6275552d-e3e0-4cb8-9a96-82e77d66c0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:55.527272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376fada-eb03-43fd-825d-e6dc8187883a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.533155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a417f5-8c65-4807-989c-ee891983f199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.534574 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a417f5-8c65-4807-989c-ee891983f199 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.534857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a417f5-8c65-4807-989c-ee891983f199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.535295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a417f5-8c65-4807-989c-ee891983f199 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.536418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a417f5-8c65-4807-989c-ee891983f199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.538689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a417f5-8c65-4807-989c-ee891983f199 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.555695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:55.600706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1088/2189] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:58:55 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:55.605687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a417f5-8c65-4807-989c-ee891983f199 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.607414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1102/2190] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:55.612013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:55.619636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275552d-e3e0-4cb8-9a96-82e77d66c0ce None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.621917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6275552d-e3e0-4cb8-9a96-82e77d66c0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:55.690795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a417f5-8c65-4807-989c-ee891983f199 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.696178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247cf04-bfd9-4263-91cc-866f8b8f748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.696696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275552d-e3e0-4cb8-9a96-82e77d66c0ce None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.696994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247cf04-bfd9-4263-91cc-866f8b8f748c admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.697650 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247cf04-bfd9-4263-91cc-866f8b8f748c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.697837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247cf04-bfd9-4263-91cc-866f8b8f748c admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.698687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247cf04-bfd9-4263-91cc-866f8b8f748c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.701199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247cf04-bfd9-4263-91cc-866f8b8f748c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.738305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1089/2191] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:58:55 2022] POST /identity/v3/users => generated 298 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:58:55.740143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247cf04-bfd9-4263-91cc-866f8b8f748c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.757374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247cf04-bfd9-4263-91cc-866f8b8f748c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.767087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d396e92-308d-4ac9-9592-659556107b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.767764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d396e92-308d-4ac9-9592-659556107b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.768109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d396e92-308d-4ac9-9592-659556107b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.768689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d396e92-308d-4ac9-9592-659556107b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.769469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d396e92-308d-4ac9-9592-659556107b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.771863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d396e92-308d-4ac9-9592-659556107b55 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.772496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e710d-6245-4dd6-a0f0-eb5949094fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.773428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e710d-6245-4dd6-a0f0-eb5949094fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.773733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e710d-6245-4dd6-a0f0-eb5949094fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.774280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e710d-6245-4dd6-a0f0-eb5949094fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.790414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1090/2192] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:55.793608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d396e92-308d-4ac9-9592-659556107b55 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.817437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d396e92-308d-4ac9-9592-659556107b55 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:55.823901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bfee2-7a90-4f61-9920-32342716232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:55.824667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bfee2-7a90-4f61-9920-32342716232b admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:55.825040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bfee2-7a90-4f61-9920-32342716232b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:55.825388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bfee2-7a90-4f61-9920-32342716232b admin admin] PATH_INFO: `/v3/projects/2f18b31685ce4a699e685f8119fb274d/users/063dbbbb6f3c4572ae81f07d79a96367/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:55.845022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bfee2-7a90-4f61-9920-32342716232b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f18b31685ce4a699e685f8119fb274d, user_id=063dbbbb6f3c4572ae81f07d79a96367, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.849950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bfee2-7a90-4f61-9920-32342716232b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.873927 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1091/2193] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:55 2022] PUT /identity/v3/projects/2f18b31685ce4a699e685f8119fb274d/users/063dbbbb6f3c4572ae81f07d79a96367/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:55.878237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bfee2-7a90-4f61-9920-32342716232b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:55.880144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e710d-6245-4dd6-a0f0-eb5949094fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:55.880906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e710d-6245-4dd6-a0f0-eb5949094fd5 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:55.894871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1103/2194] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:58:56.050566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bfee2-7a90-4f61-9920-32342716232b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:56.060844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3ca4b-7f99-422f-8e24-1e7f140815b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:56.062477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3ca4b-7f99-422f-8e24-1e7f140815b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:56.063704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3ca4b-7f99-422f-8e24-1e7f140815b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:56.064133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3ca4b-7f99-422f-8e24-1e7f140815b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:56.068000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3ca4b-7f99-422f-8e24-1e7f140815b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:56.070012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3ca4b-7f99-422f-8e24-1e7f140815b0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:56.100594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1092/2195] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:58:56.102267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e710d-6245-4dd6-a0f0-eb5949094fd5 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:56.116351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e710d-6245-4dd6-a0f0-eb5949094fd5 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:56.139612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f906b-ab80-4d1f-8350-b5e7ce4acdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:56.140679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f906b-ab80-4d1f-8350-b5e7ce4acdab admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:56.141055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f906b-ab80-4d1f-8350-b5e7ce4acdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:56.141337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f906b-ab80-4d1f-8350-b5e7ce4acdab admin admin] PATH_INFO: `/v3/projects/2f18b31685ce4a699e685f8119fb274d/users/063dbbbb6f3c4572ae81f07d79a96367/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:56.146786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f906b-ab80-4d1f-8350-b5e7ce4acdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f18b31685ce4a699e685f8119fb274d, user_id=063dbbbb6f3c4572ae81f07d79a96367, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:56.151252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f906b-ab80-4d1f-8350-b5e7ce4acdab admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:56.178677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1104/2196] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:58:56 2022] PUT /identity/v3/projects/2f18b31685ce4a699e685f8119fb274d/users/063dbbbb6f3c4572ae81f07d79a96367/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:58:56.179598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34fd42-9525-42d2-ab61-af5069ed68d9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:56.180067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34fd42-9525-42d2-ab61-af5069ed68d9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:56.180349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34fd42-9525-42d2-ab61-af5069ed68d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:56.225858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4d34fd42-9525-42d2-ab61-af5069ed68d9 None None] MFA Rules not processed for user `063dbbbb6f3c4572ae81f07d79a96367`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:56.227259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4d34fd42-9525-42d2-ab61-af5069ed68d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:56.323509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:56.387933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1093/2197] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:58:56.406537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f906b-ab80-4d1f-8350-b5e7ce4acdab admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:56.481034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:58:56.532767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f906b-ab80-4d1f-8350-b5e7ce4acdab admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:56.548348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649cc46-b349-4c41-86cf-d95991286974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:56.549172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649cc46-b349-4c41-86cf-d95991286974 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:56.550093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649cc46-b349-4c41-86cf-d95991286974 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:56.550426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649cc46-b349-4c41-86cf-d95991286974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:56.625902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649cc46-b349-4c41-86cf-d95991286974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:56.627609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649cc46-b349-4c41-86cf-d95991286974 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:56.639029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1105/2198] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:58:58.641621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddade264-8a3b-4860-a43b-6bc0eece7392 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:58.642874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddade264-8a3b-4860-a43b-6bc0eece7392 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:58.644469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddade264-8a3b-4860-a43b-6bc0eece7392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:58.671892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ddade264-8a3b-4860-a43b-6bc0eece7392 None None] MFA Rules not processed for user `c0da8726d48b42f0bc70ebcd4b32c887`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:58:58.674375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ddade264-8a3b-4860-a43b-6bc0eece7392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:58.760785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1094/2199] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:58:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:58:59.324030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649cc46-b349-4c41-86cf-d95991286974 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:58:59.325771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-1649cc46-b349-4c41-86cf-d95991286974 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:58:59.338279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649cc46-b349-4c41-86cf-d95991286974 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:58:59.342830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c521f-91a2-4129-8188-5dffb57f1196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:58:59.343629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c521f-91a2-4129-8188-5dffb57f1196 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:58:59.343947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c521f-91a2-4129-8188-5dffb57f1196 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:58:59.344222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c521f-91a2-4129-8188-5dffb57f1196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:58:59.415228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c521f-91a2-4129-8188-5dffb57f1196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:58:59.416733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c521f-91a2-4129-8188-5dffb57f1196 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:58:59.428955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1106/2200] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:58:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:59:00.118314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddade264-8a3b-4860-a43b-6bc0eece7392 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:00.131178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddade264-8a3b-4860-a43b-6bc0eece7392 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:00.135287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b559df-32d2-4878-b1da-be924bb8eb7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:00.135892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b559df-32d2-4878-b1da-be924bb8eb7b service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:00.136179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b559df-32d2-4878-b1da-be924bb8eb7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:00.136458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b559df-32d2-4878-b1da-be924bb8eb7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:00.146373 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b559df-32d2-4878-b1da-be924bb8eb7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:00.147822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b559df-32d2-4878-b1da-be924bb8eb7b service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:00.160880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1095/2201] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:59:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:59:05.694840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c521f-91a2-4129-8188-5dffb57f1196 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:05.884476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c521f-91a2-4129-8188-5dffb57f1196 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:05.889647 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:05.890683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:05.890977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:05.895150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] PATH_INFO: `/v3/users/b3768ea847d34de188db083379b0e1e5` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:05.897448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3768ea847d34de188db083379b0e1e5)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:05.901183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:05.963758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:06.004603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3768ea847d34de188db083379b0e1e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:06.005372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3768ea847d34de188db083379b0e1e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:06.020603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3768ea847d34de188db083379b0e1e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:06.021347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3768ea847d34de188db083379b0e1e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:06.036743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3768ea847d34de188db083379b0e1e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:06.088054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1107/2202] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:59:05 2022] DELETE /identity/v3/users/b3768ea847d34de188db083379b0e1e5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:59:06.092714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87925625-50db-441d-a10c-6df23eeef2d8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:06.093259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87925625-50db-441d-a10c-6df23eeef2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:06.093579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87925625-50db-441d-a10c-6df23eeef2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:06.125843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-87925625-50db-441d-a10c-6df23eeef2d8 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:06.127575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-87925625-50db-441d-a10c-6df23eeef2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:06.250710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1096/2203] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:06.261837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:06.263349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:06.336070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e085a-1218-49d8-bd56-fa9a854a075c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:06.352550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b0ba4-8134-4b06-9af6-6e5976651e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:06.353532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b0ba4-8134-4b06-9af6-6e5976651e94 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:06.353949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b0ba4-8134-4b06-9af6-6e5976651e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:06.354328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b0ba4-8134-4b06-9af6-6e5976651e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:06.374454 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b0ba4-8134-4b06-9af6-6e5976651e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:06.376154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b0ba4-8134-4b06-9af6-6e5976651e94 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:06.399489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1108/2204] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:59:06.833953 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87925625-50db-441d-a10c-6df23eeef2d8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:06.835431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-87925625-50db-441d-a10c-6df23eeef2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:06.858302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87925625-50db-441d-a10c-6df23eeef2d8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:06.866720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:06.868046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:06.868046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:06.868539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] PATH_INFO: `/v3/projects/c699c5bed83549a0be701b20ead4023c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:06.870431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c699c5bed83549a0be701b20ead4023c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:06.876229 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:06.950809 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c699c5bed83549a0be701b20ead4023c', 'request_id': 'req-605b9b34-fb17-4a52-9464-383f8bed6b5a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:06.968395 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c699c5bed83549a0be701b20ead4023c', 'request_id': 'req-605b9b34-fb17-4a52-9464-383f8bed6b5a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:06.987510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c699c5bed83549a0be701b20ead4023c', 'request_id': 'req-605b9b34-fb17-4a52-9464-383f8bed6b5a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:06.990343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:07.043252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] The token cache is being invalidate because project c699c5bed83549a0be701b20ead4023c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:59:07.043714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.052089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.058507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1097/2205] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:59:06 2022] DELETE /identity/v3/projects/c699c5bed83549a0be701b20ead4023c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:59:07.062146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b0ba4-8134-4b06-9af6-6e5976651e94 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:07.075991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b0ba4-8134-4b06-9af6-6e5976651e94 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:07.158605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:07.160016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:07.160642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:07.161009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] PATH_INFO: `/v3/users/ec4e9cb7826d47f5908a81dbfea7b02a` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:07.162315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4e9cb7826d47f5908a81dbfea7b02a)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:07.164287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:07.233565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4e9cb7826d47f5908a81dbfea7b02a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.234667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4e9cb7826d47f5908a81dbfea7b02a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.237702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:07.247503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4e9cb7826d47f5908a81dbfea7b02a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.248130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4e9cb7826d47f5908a81dbfea7b02a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.262552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4e9cb7826d47f5908a81dbfea7b02a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.289785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1109/2206] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:59:07 2022] DELETE /identity/v3/users/ec4e9cb7826d47f5908a81dbfea7b02a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:59:07.693603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:07.695542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:07.712940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9b34-fb17-4a52-9464-383f8bed6b5a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:07.814290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:07.816862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:07.818804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:07.819573 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] PATH_INFO: `/v3/projects/fa37a295342042dca113e316e643af20` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:07.823228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa37a295342042dca113e316e643af20)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:07.825378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:07.891887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa37a295342042dca113e316e643af20', 'request_id': 'req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.898635 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa37a295342042dca113e316e643af20', 'request_id': 'req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.908594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa37a295342042dca113e316e643af20', 'request_id': 'req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.921075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] The token cache is being invalidate because project fa37a295342042dca113e316e643af20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:59:07.921459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.922124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:07.924405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1098/2207] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:59:07 2022] DELETE /identity/v3/projects/fa37a295342042dca113e316e643af20 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:59:07.928234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:07.929834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:07.944030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9be110-5c21-488e-8ed8-ddc21ecccecd admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:08.025082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:08.026188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:08.026576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:08.026957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] PATH_INFO: `/v3/users/063dbbbb6f3c4572ae81f07d79a96367` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:08.028435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=063dbbbb6f3c4572ae81f07d79a96367)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:08.030982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:08.123702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '063dbbbb6f3c4572ae81f07d79a96367'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.129493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '063dbbbb6f3c4572ae81f07d79a96367'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.146822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '063dbbbb6f3c4572ae81f07d79a96367'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.148023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '063dbbbb6f3c4572ae81f07d79a96367'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.173381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-16c5d0cd-ed7a-46cd-a38c-432018052f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '063dbbbb6f3c4572ae81f07d79a96367'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.196382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1110/2208] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:59:07 2022] DELETE /identity/v3/users/063dbbbb6f3c4572ae81f07d79a96367 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:59:08.550484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:08.551930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:08.584837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ce134-b3ff-46dc-92d5-265143dbf7c1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:08.650120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:08.650790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:08.651069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:08.651337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] PATH_INFO: `/v3/projects/2f18b31685ce4a699e685f8119fb274d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:08.652474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f18b31685ce4a699e685f8119fb274d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:08.654494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:08.696608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:08.718124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f18b31685ce4a699e685f8119fb274d', 'request_id': 'req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.724954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f18b31685ce4a699e685f8119fb274d', 'request_id': 'req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.738330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f18b31685ce4a699e685f8119fb274d', 'request_id': 'req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.753459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] The token cache is being invalidate because project 2f18b31685ce4a699e685f8119fb274d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:59:08.753933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.754806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d05cfce0-0bfc-4acd-957d-5c8da3c6d479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:08.756736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1099/2209] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:59:08 2022] DELETE /identity/v3/projects/2f18b31685ce4a699e685f8119fb274d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:59:08.761700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28c7a1-de58-41ab-a797-c72ae26ef789 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:08.762202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28c7a1-de58-41ab-a797-c72ae26ef789 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:08.762480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28c7a1-de58-41ab-a797-c72ae26ef789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:08.786979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3c28c7a1-de58-41ab-a797-c72ae26ef789 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:08.788272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3c28c7a1-de58-41ab-a797-c72ae26ef789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:08.838255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:08.864911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1111/2210] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:59:08.875417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc364878-a31d-4b9e-97e8-c70071ce0d6e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:08.876089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc364878-a31d-4b9e-97e8-c70071ce0d6e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:08.876683 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc364878-a31d-4b9e-97e8-c70071ce0d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:08.916129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cc364878-a31d-4b9e-97e8-c70071ce0d6e None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:08.917489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cc364878-a31d-4b9e-97e8-c70071ce0d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:08.942917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1100/2211] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:08.949568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28c7a1-de58-41ab-a797-c72ae26ef789 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:08.961269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28c7a1-de58-41ab-a797-c72ae26ef789 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:08.966820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a3b01-b6ac-4659-bbb6-148d959b07d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:08.967491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a3b01-b6ac-4659-bbb6-148d959b07d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:08.967875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a3b01-b6ac-4659-bbb6-148d959b07d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:08.968323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a3b01-b6ac-4659-bbb6-148d959b07d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:08.969056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a3b01-b6ac-4659-bbb6-148d959b07d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:59:08.969482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a3b01-b6ac-4659-bbb6-148d959b07d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:08.973808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a3b01-b6ac-4659-bbb6-148d959b07d6 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:08.990314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1112/2212] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:59:08.996302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc364878-a31d-4b9e-97e8-c70071ce0d6e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:09.006224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc364878-a31d-4b9e-97e8-c70071ce0d6e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:09.011119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6ba91-03ae-4c6d-a028-00f282bcc541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:09.011832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6ba91-03ae-4c6d-a028-00f282bcc541 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:09.012009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6ba91-03ae-4c6d-a028-00f282bcc541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:09.012284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6ba91-03ae-4c6d-a028-00f282bcc541 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:09.012978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6ba91-03ae-4c6d-a028-00f282bcc541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:09.014894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6ba91-03ae-4c6d-a028-00f282bcc541 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:09.059672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1101/2213] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:59:08 2022] POST /identity/v3/projects => generated 391 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:59:09.066045 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a3b01-b6ac-4659-bbb6-148d959b07d6 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:09.077278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a3b01-b6ac-4659-bbb6-148d959b07d6 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:09.149928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e06d1-fde9-42fa-a3da-38f9b045dedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:09.151421 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e06d1-fde9-42fa-a3da-38f9b045dedb admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:09.151881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e06d1-fde9-42fa-a3da-38f9b045dedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:09.152294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e06d1-fde9-42fa-a3da-38f9b045dedb admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:09.153122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e06d1-fde9-42fa-a3da-38f9b045dedb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:09.155225 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e06d1-fde9-42fa-a3da-38f9b045dedb admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:09.189506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1113/2214] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:59:09 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:59:09.195356 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6ba91-03ae-4c6d-a028-00f282bcc541 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:09.207486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6ba91-03ae-4c6d-a028-00f282bcc541 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:09.215905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee697fc-33d3-4cb6-b737-ae0e5203fe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:09.216535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee697fc-33d3-4cb6-b737-ae0e5203fe36 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:09.216862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee697fc-33d3-4cb6-b737-ae0e5203fe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:09.217122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee697fc-33d3-4cb6-b737-ae0e5203fe36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:09.217828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee697fc-33d3-4cb6-b737-ae0e5203fe36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:09.221714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee697fc-33d3-4cb6-b737-ae0e5203fe36 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:09.231513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1102/2215] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:59:09.235545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e06d1-fde9-42fa-a3da-38f9b045dedb admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:09.254975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e06d1-fde9-42fa-a3da-38f9b045dedb admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:09.270182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b5179-caca-47b0-918e-6dd457240516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:09.270910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b5179-caca-47b0-918e-6dd457240516 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:09.271271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b5179-caca-47b0-918e-6dd457240516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:09.271621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b5179-caca-47b0-918e-6dd457240516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:09.272376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b5179-caca-47b0-918e-6dd457240516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:09.274328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b5179-caca-47b0-918e-6dd457240516 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:09.295066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1114/2216] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:59:09.301447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee697fc-33d3-4cb6-b737-ae0e5203fe36 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:09.324099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee697fc-33d3-4cb6-b737-ae0e5203fe36 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:09.331591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6a8c0-94af-4f90-b2df-ad4a7446993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:09.332397 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6a8c0-94af-4f90-b2df-ad4a7446993e admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:09.332850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6a8c0-94af-4f90-b2df-ad4a7446993e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:09.333199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6a8c0-94af-4f90-b2df-ad4a7446993e admin admin] PATH_INFO: `/v3/projects/ea927da843db427bbac553890d9dcfe2/users/c9d7b4f572b0429b884eb37bc4ad88fc/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:09.350327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6a8c0-94af-4f90-b2df-ad4a7446993e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea927da843db427bbac553890d9dcfe2, user_id=c9d7b4f572b0429b884eb37bc4ad88fc, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:09.353243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6a8c0-94af-4f90-b2df-ad4a7446993e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:09.371489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1103/2217] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:59:09 2022] PUT /identity/v3/projects/ea927da843db427bbac553890d9dcfe2/users/c9d7b4f572b0429b884eb37bc4ad88fc/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:59:09.383151 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617eac-917a-4fad-8d41-f917bb141399 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:09.383590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617eac-917a-4fad-8d41-f917bb141399 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:09.383970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617eac-917a-4fad-8d41-f917bb141399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:09.438457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-73617eac-917a-4fad-8d41-f917bb141399 None None] MFA Rules not processed for user `c9d7b4f572b0429b884eb37bc4ad88fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:09.440606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-73617eac-917a-4fad-8d41-f917bb141399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:09.523232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1115/2218] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:09.532736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6a8c0-94af-4f90-b2df-ad4a7446993e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:09.535493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d3b6a8c0-94af-4f90-b2df-ad4a7446993e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:09.608617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6a8c0-94af-4f90-b2df-ad4a7446993e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:09.613636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63ecf3-9acf-4f24-8d0a-3d64f91ffda5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:09.614293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63ecf3-9acf-4f24-8d0a-3d64f91ffda5 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:09.614685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63ecf3-9acf-4f24-8d0a-3d64f91ffda5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:09.614812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63ecf3-9acf-4f24-8d0a-3d64f91ffda5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:09.624419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63ecf3-9acf-4f24-8d0a-3d64f91ffda5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:09.625915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63ecf3-9acf-4f24-8d0a-3d64f91ffda5 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:09.636452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1104/2219] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:59:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:59:12.490046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ec5d5-a16a-44ce-a8f8-cc8860c27b10 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:12.490566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ec5d5-a16a-44ce-a8f8-cc8860c27b10 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:12.490665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ec5d5-a16a-44ce-a8f8-cc8860c27b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:12.514514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5f3ec5d5-a16a-44ce-a8f8-cc8860c27b10 None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:12.516258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5f3ec5d5-a16a-44ce-a8f8-cc8860c27b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:12.565862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:12.593749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1116/2220] 10.222.0.50 () {58 vars in 967 bytes} [Thu Sep 15 10:59:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:12.607522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63ecf3-9acf-4f24-8d0a-3d64f91ffda5 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:12.610070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5e63ecf3-9acf-4f24-8d0a-3d64f91ffda5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:12.672912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:12.694849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63ecf3-9acf-4f24-8d0a-3d64f91ffda5 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:12.700107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7b4d4-a945-470c-953a-7217d1aaea97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:12.700718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7b4d4-a945-470c-953a-7217d1aaea97 service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:12.701034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7b4d4-a945-470c-953a-7217d1aaea97 service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:12.701306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7b4d4-a945-470c-953a-7217d1aaea97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:12.709804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7b4d4-a945-470c-953a-7217d1aaea97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:12.711402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7b4d4-a945-470c-953a-7217d1aaea97 service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:12.719576 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1105/2221] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 10:59:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 10:59:17.240903 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364406-b70a-49c4-80bc-2110ce2d597c None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:17.241404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364406-b70a-49c4-80bc-2110ce2d597c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:17.241508 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364406-b70a-49c4-80bc-2110ce2d597c None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:17.242779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1117/2222] 10.222.0.50 () {54 vars in 961 bytes} [Thu Sep 15 10:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 10:59:17.247672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c8192-d015-416a-a906-8e8fcbf14962 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:17.248080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c8192-d015-416a-a906-8e8fcbf14962 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:17.248238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c8192-d015-416a-a906-8e8fcbf14962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:17.267874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b72c8192-d015-416a-a906-8e8fcbf14962 None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:17.269121 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b72c8192-d015-416a-a906-8e8fcbf14962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:17.336174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1106/2223] 10.222.0.50 () {58 vars in 1058 bytes} [Thu Sep 15 10:59:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:17.344926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364406-b70a-49c4-80bc-2110ce2d597c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:17.359854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364406-b70a-49c4-80bc-2110ce2d597c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:17.365855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ffba-819c-4a75-a5e9-d6e1d5bd2329 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:17.366467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ffba-819c-4a75-a5e9-d6e1d5bd2329 service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:17.366729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ffba-819c-4a75-a5e9-d6e1d5bd2329 service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:17.366999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ffba-819c-4a75-a5e9-d6e1d5bd2329 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:17.368700 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-daa4ffba-819c-4a75-a5e9-d6e1d5bd2329 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:17.380213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4ffba-819c-4a75-a5e9-d6e1d5bd2329 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:17.381835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4ffba-819c-4a75-a5e9-d6e1d5bd2329 service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:17.391493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1118/2224] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:59:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:59:18.965142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c8192-d015-416a-a906-8e8fcbf14962 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:18.967107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b72c8192-d015-416a-a906-8e8fcbf14962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:19.046595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c8192-d015-416a-a906-8e8fcbf14962 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:19.053072 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:19.053754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.054036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.054310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] PATH_INFO: `/v3/users/c1c8a563a78546fe9714d69913418c6b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.055524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c8a563a78546fe9714d69913418c6b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:19.057553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:19.110161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c8a563a78546fe9714d69913418c6b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.118632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c8a563a78546fe9714d69913418c6b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.119319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c8a563a78546fe9714d69913418c6b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.127725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c8a563a78546fe9714d69913418c6b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.128453 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-66b7917a-2c9b-4f17-9240-9baecef85507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c8a563a78546fe9714d69913418c6b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.142621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:19.148379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1107/2225] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:59:18 2022] DELETE /identity/v3/users/c1c8a563a78546fe9714d69913418c6b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:59:19.338566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ffba-819c-4a75-a5e9-d6e1d5bd2329 service placement] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:19.350694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ffba-819c-4a75-a5e9-d6e1d5bd2329 service placement] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:19.411780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:19.412440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.412718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.413033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] PATH_INFO: `/v3/projects/47362556b66f405fbedb849e6d0f0b9f` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.414117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47362556b66f405fbedb849e6d0f0b9f)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:19.416019 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:19.461839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47362556b66f405fbedb849e6d0f0b9f', 'request_id': 'req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.467391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47362556b66f405fbedb849e6d0f0b9f', 'request_id': 'req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.474480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47362556b66f405fbedb849e6d0f0b9f', 'request_id': 'req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.485612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] The token cache is being invalidate because project 47362556b66f405fbedb849e6d0f0b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:59:19.485904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.486535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8a71f057-12d9-4767-bbb6-1ccbc9fedd34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:19.488037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1119/2226] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:59:19 2022] DELETE /identity/v3/projects/47362556b66f405fbedb849e6d0f0b9f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:59:19.493350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baafad-f889-4744-8e32-1cbe7644bdb3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.493698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baafad-f889-4744-8e32-1cbe7644bdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.494024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baafad-f889-4744-8e32-1cbe7644bdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.514950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c8baafad-f889-4744-8e32-1cbe7644bdb3 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:19.516183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c8baafad-f889-4744-8e32-1cbe7644bdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:19.581208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1108/2227] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:19.589598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da18290-4c5a-4ad4-a42a-3dee5db12047 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.589886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da18290-4c5a-4ad4-a42a-3dee5db12047 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.590157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da18290-4c5a-4ad4-a42a-3dee5db12047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.610099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7da18290-4c5a-4ad4-a42a-3dee5db12047 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:19.611710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7da18290-4c5a-4ad4-a42a-3dee5db12047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:19.620307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1120/2228] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:59:19.624511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baafad-f889-4744-8e32-1cbe7644bdb3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:19.635648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baafad-f889-4744-8e32-1cbe7644bdb3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:19.640219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdbb77-de42-4eff-9a34-43228cd7234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:19.640850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdbb77-de42-4eff-9a34-43228cd7234d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.641135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdbb77-de42-4eff-9a34-43228cd7234d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.641403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdbb77-de42-4eff-9a34-43228cd7234d admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.642619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdbb77-de42-4eff-9a34-43228cd7234d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:59:19.642917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdbb77-de42-4eff-9a34-43228cd7234d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:19.644514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdbb77-de42-4eff-9a34-43228cd7234d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:19.656086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1109/2229] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 10:59:19.660282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da18290-4c5a-4ad4-a42a-3dee5db12047 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:19.670091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da18290-4c5a-4ad4-a42a-3dee5db12047 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:19.674874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f474043-b2a3-4e23-a2d7-f3e6414e3151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:19.675697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f474043-b2a3-4e23-a2d7-f3e6414e3151 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.675975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f474043-b2a3-4e23-a2d7-f3e6414e3151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.676247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f474043-b2a3-4e23-a2d7-f3e6414e3151 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.677039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f474043-b2a3-4e23-a2d7-f3e6414e3151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:19.678832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f474043-b2a3-4e23-a2d7-f3e6414e3151 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:19.706898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1121/2230] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:59:19 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:59:19.710993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdbb77-de42-4eff-9a34-43228cd7234d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:19.721391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdbb77-de42-4eff-9a34-43228cd7234d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:19.777642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758b454-2e34-4dfb-97a7-665c20d4646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:19.778283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758b454-2e34-4dfb-97a7-665c20d4646b admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.778572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758b454-2e34-4dfb-97a7-665c20d4646b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.778851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758b454-2e34-4dfb-97a7-665c20d4646b admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.779490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758b454-2e34-4dfb-97a7-665c20d4646b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:19.781507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758b454-2e34-4dfb-97a7-665c20d4646b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:19.802834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1110/2231] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:59:19 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:59:19.806957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f474043-b2a3-4e23-a2d7-f3e6414e3151 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:19.818099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f474043-b2a3-4e23-a2d7-f3e6414e3151 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:19.822914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeed7be-4224-4eb6-8dfa-659f18326552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:19.823502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeed7be-4224-4eb6-8dfa-659f18326552 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.823776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeed7be-4224-4eb6-8dfa-659f18326552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.824044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeed7be-4224-4eb6-8dfa-659f18326552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.824690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeed7be-4224-4eb6-8dfa-659f18326552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:19.826319 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeed7be-4224-4eb6-8dfa-659f18326552 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:19.836890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1122/2232] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:59:19.840699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758b454-2e34-4dfb-97a7-665c20d4646b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:19.850715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758b454-2e34-4dfb-97a7-665c20d4646b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:19.855367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3c71d-cca7-4022-bc97-ab12af9533e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:19.855962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3c71d-cca7-4022-bc97-ab12af9533e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.856246 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3c71d-cca7-4022-bc97-ab12af9533e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.856522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3c71d-cca7-4022-bc97-ab12af9533e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.857201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3c71d-cca7-4022-bc97-ab12af9533e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:19.858849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3c71d-cca7-4022-bc97-ab12af9533e0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:19.869915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1111/2233] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:59:19.873909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeed7be-4224-4eb6-8dfa-659f18326552 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:19.884575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeed7be-4224-4eb6-8dfa-659f18326552 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:19.889360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86055c-0677-4ac3-8c54-a769111a0803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:19.890001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86055c-0677-4ac3-8c54-a769111a0803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.890284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86055c-0677-4ac3-8c54-a769111a0803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.890566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86055c-0677-4ac3-8c54-a769111a0803 admin admin] PATH_INFO: `/v3/projects/1c54ca425ddf47a3b6b79f3a6baa6e2d/users/d1e32ef95ff84d01881bd418bad9aaa3/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.905033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86055c-0677-4ac3-8c54-a769111a0803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c54ca425ddf47a3b6b79f3a6baa6e2d, user_id=d1e32ef95ff84d01881bd418bad9aaa3, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:19.907712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86055c-0677-4ac3-8c54-a769111a0803 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:19.916263 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:19.925059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1123/2234] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:59:19 2022] PUT /identity/v3/projects/1c54ca425ddf47a3b6b79f3a6baa6e2d/users/d1e32ef95ff84d01881bd418bad9aaa3/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:59:19.935357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ed793-5b6f-47d5-bb70-1720c8d64c36 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:19.935649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ed793-5b6f-47d5-bb70-1720c8d64c36 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:19.935931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ed793-5b6f-47d5-bb70-1720c8d64c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:19.963796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-039ed793-5b6f-47d5-bb70-1720c8d64c36 None None] MFA Rules not processed for user `d1e32ef95ff84d01881bd418bad9aaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:19.965038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-039ed793-5b6f-47d5-bb70-1720c8d64c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:20.018472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:20.029109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1112/2235] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:20.040742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86055c-0677-4ac3-8c54-a769111a0803 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:20.042536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-af86055c-0677-4ac3-8c54-a769111a0803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:20.100053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86055c-0677-4ac3-8c54-a769111a0803 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:20.103756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febf62b-57b9-4c42-ae31-395a18ce1558 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:20.104359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf62b-57b9-4c42-ae31-395a18ce1558 service glance] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:20.104641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf62b-57b9-4c42-ae31-395a18ce1558 service glance] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:20.104956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf62b-57b9-4c42-ae31-395a18ce1558 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:20.113739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febf62b-57b9-4c42-ae31-395a18ce1558 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:20.115179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febf62b-57b9-4c42-ae31-395a18ce1558 service glance] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:20.123962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1124/2236] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:59:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:59:30.029437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:30.119917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=27 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:39.610596 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ed793-5b6f-47d5-bb70-1720c8d64c36 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:39.612453 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-039ed793-5b6f-47d5-bb70-1720c8d64c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:39.700907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ed793-5b6f-47d5-bb70-1720c8d64c36 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:39.706612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:39.707296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:39.707568 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:39.707837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] PATH_INFO: `/v3/users/c9d7b4f572b0429b884eb37bc4ad88fc` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:39.709625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9d7b4f572b0429b884eb37bc4ad88fc)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:39.711815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:39.740045 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d4dd-90f7-42f5-818c-487b4373266a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:39.740704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d4dd-90f7-42f5-818c-487b4373266a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:39.740704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d4dd-90f7-42f5-818c-487b4373266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:39.762034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4a82d4dd-90f7-42f5-818c-487b4373266a None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:39.763246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4a82d4dd-90f7-42f5-818c-487b4373266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:39.766470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9d7b4f572b0429b884eb37bc4ad88fc'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:39.774364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9d7b4f572b0429b884eb37bc4ad88fc'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:39.775119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9d7b4f572b0429b884eb37bc4ad88fc'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:39.783129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9d7b4f572b0429b884eb37bc4ad88fc'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:39.783788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-473b1c97-b45f-4a7d-9e5f-40f99488fe91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9d7b4f572b0429b884eb37bc4ad88fc'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:39.803305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1113/2237] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 10:59:39 2022] DELETE /identity/v3/users/c9d7b4f572b0429b884eb37bc4ad88fc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 10:59:39.807629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8947542-cb6f-40a1-8545-1d4fbcfe5fd4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:39.807915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8947542-cb6f-40a1-8545-1d4fbcfe5fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:39.808243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8947542-cb6f-40a1-8545-1d4fbcfe5fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:39.821952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1125/2238] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 10:59:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:39.827469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f8947542-cb6f-40a1-8545-1d4fbcfe5fd4 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:39.829570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f8947542-cb6f-40a1-8545-1d4fbcfe5fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:39.835196 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d4dd-90f7-42f5-818c-487b4373266a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:39.837410 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4a82d4dd-90f7-42f5-818c-487b4373266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:39.891719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1114/2239] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:59:39.901158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8947542-cb6f-40a1-8545-1d4fbcfe5fd4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:39.902470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f8947542-cb6f-40a1-8545-1d4fbcfe5fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:39.908566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d4dd-90f7-42f5-818c-487b4373266a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:39.927593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:39.970038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8947542-cb6f-40a1-8545-1d4fbcfe5fd4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:39.973888 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1cc0c-9543-49cc-90da-c9922aad10c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:39.974270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04ac5e-b712-4454-9b6c-397a1f0134de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:39.974498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1cc0c-9543-49cc-90da-c9922aad10c6 service swift] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:39.974777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1cc0c-9543-49cc-90da-c9922aad10c6 service swift] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:39.974894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04ac5e-b712-4454-9b6c-397a1f0134de service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:39.975044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1cc0c-9543-49cc-90da-c9922aad10c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:39.975155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04ac5e-b712-4454-9b6c-397a1f0134de service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:39.975397 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04ac5e-b712-4454-9b6c-397a1f0134de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:39.984509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04ac5e-b712-4454-9b6c-397a1f0134de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:39.985809 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04ac5e-b712-4454-9b6c-397a1f0134de service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:39.996415 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1115/2240] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 10:59:40.028016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:40.040177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1cc0c-9543-49cc-90da-c9922aad10c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:40.041927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1cc0c-9543-49cc-90da-c9922aad10c6 service swift] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:40.050648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1126/2241] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 10:59:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 10:59:40.289287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04ac5e-b712-4454-9b6c-397a1f0134de service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:40.291201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fa04ac5e-b712-4454-9b6c-397a1f0134de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:40.306232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04ac5e-b712-4454-9b6c-397a1f0134de service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:40.311217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:40.312463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.312754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.313240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] PATH_INFO: `/v3/projects/ea927da843db427bbac553890d9dcfe2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.314492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea927da843db427bbac553890d9dcfe2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:40.316442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:40.368844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea927da843db427bbac553890d9dcfe2', 'request_id': 'req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:40.374778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea927da843db427bbac553890d9dcfe2', 'request_id': 'req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:40.383670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea927da843db427bbac553890d9dcfe2', 'request_id': 'req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:40.396188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] The token cache is being invalidate because project ea927da843db427bbac553890d9dcfe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 10:59:40.396481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:40.397137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-140a0df6-9e2b-4dd8-a2e8-9748aac4a45f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 10:59:40.398708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1116/2242] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 10:59:40 2022] DELETE /identity/v3/projects/ea927da843db427bbac553890d9dcfe2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:59:40.403436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da7600-1e9a-415f-ae40-fd9ef8c24637 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.403726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da7600-1e9a-415f-ae40-fd9ef8c24637 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.404020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da7600-1e9a-415f-ae40-fd9ef8c24637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.425105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-27da7600-1e9a-415f-ae40-fd9ef8c24637 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:40.426307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-27da7600-1e9a-415f-ae40-fd9ef8c24637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:40.489758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1127/2243] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:40.497489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54232d08-9d67-40c5-aa9a-30e440129720 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.497780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54232d08-9d67-40c5-aa9a-30e440129720 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.498056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54232d08-9d67-40c5-aa9a-30e440129720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.518702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-54232d08-9d67-40c5-aa9a-30e440129720 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:40.521990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-54232d08-9d67-40c5-aa9a-30e440129720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:40.530644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1117/2244] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:40.534742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da7600-1e9a-415f-ae40-fd9ef8c24637 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:40.548563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da7600-1e9a-415f-ae40-fd9ef8c24637 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:40.557919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b63c7-06fa-4adc-837d-1457327e6d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:40.558509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b63c7-06fa-4adc-837d-1457327e6d79 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.559485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b63c7-06fa-4adc-837d-1457327e6d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.559485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b63c7-06fa-4adc-837d-1457327e6d79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.559724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b63c7-06fa-4adc-837d-1457327e6d79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 10:59:40.560395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b63c7-06fa-4adc-837d-1457327e6d79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:40.564083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b63c7-06fa-4adc-837d-1457327e6d79 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:40.579653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1128/2245] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 10:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 10:59:40.584470 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54232d08-9d67-40c5-aa9a-30e440129720 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:40.595633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54232d08-9d67-40c5-aa9a-30e440129720 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:40.600469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbe595-08ce-42f3-9c75-cd2749574564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:40.601070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbe595-08ce-42f3-9c75-cd2749574564 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.601361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbe595-08ce-42f3-9c75-cd2749574564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.601656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbe595-08ce-42f3-9c75-cd2749574564 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.602244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbe595-08ce-42f3-9c75-cd2749574564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:40.604149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbe595-08ce-42f3-9c75-cd2749574564 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:40.630035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109411d-779d-4851-8117-986444658ea5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.630328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109411d-779d-4851-8117-986444658ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.630637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109411d-779d-4851-8117-986444658ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.638299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1118/2246] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 10:59:40 2022] POST /identity/v3/projects => generated 397 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 10:59:40.639305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbe595-08ce-42f3-9c75-cd2749574564 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:40.656849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-c109411d-779d-4851-8117-986444658ea5 None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:40.658169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c109411d-779d-4851-8117-986444658ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:40.721280 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbe595-08ce-42f3-9c75-cd2749574564 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:40.726167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1129/2247] 10.222.0.50 () {58 vars in 967 bytes} [Thu Sep 15 10:59:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 10:59:40.726382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2b9d3-e106-476c-80f1-ca162dc14751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:40.726993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2b9d3-e106-476c-80f1-ca162dc14751 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.727355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2b9d3-e106-476c-80f1-ca162dc14751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.727629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2b9d3-e106-476c-80f1-ca162dc14751 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.728252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2b9d3-e106-476c-80f1-ca162dc14751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:40.730199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2b9d3-e106-476c-80f1-ca162dc14751 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:40.740442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109411d-779d-4851-8117-986444658ea5 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:40.742468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c109411d-779d-4851-8117-986444658ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:40.766233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1119/2248] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 10:59:40 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 10:59:40.771254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2b9d3-e106-476c-80f1-ca162dc14751 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:40.782817 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2b9d3-e106-476c-80f1-ca162dc14751 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:40.788985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc3e14-2fc2-4d38-a448-9bb0675ec6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:40.789740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc3e14-2fc2-4d38-a448-9bb0675ec6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.790022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc3e14-2fc2-4d38-a448-9bb0675ec6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.790294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc3e14-2fc2-4d38-a448-9bb0675ec6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.790943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc3e14-2fc2-4d38-a448-9bb0675ec6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:40.792673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc3e14-2fc2-4d38-a448-9bb0675ec6c6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:40.794171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:40.810037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1120/2249] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 10:59:40.814468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc3e14-2fc2-4d38-a448-9bb0675ec6c6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:40.824615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc3e14-2fc2-4d38-a448-9bb0675ec6c6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:40.829484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b894d1f-3506-46ef-885d-8e07738439e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:40.829944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109411d-779d-4851-8117-986444658ea5 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:40.830065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b894d1f-3506-46ef-885d-8e07738439e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.830353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b894d1f-3506-46ef-885d-8e07738439e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.830658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b894d1f-3506-46ef-885d-8e07738439e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.831289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b894d1f-3506-46ef-885d-8e07738439e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:40.832959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b894d1f-3506-46ef-885d-8e07738439e7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:40.835076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba375e7d-3aa5-4217-9528-e6dca46a0a62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:40.835683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba375e7d-3aa5-4217-9528-e6dca46a0a62 service swift] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.835961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba375e7d-3aa5-4217-9528-e6dca46a0a62 service swift] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.836228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba375e7d-3aa5-4217-9528-e6dca46a0a62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.843966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1121/2250] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 10:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 10:59:40.844413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba375e7d-3aa5-4217-9528-e6dca46a0a62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:40.846061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba375e7d-3aa5-4217-9528-e6dca46a0a62 service swift] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:40.848001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b894d1f-3506-46ef-885d-8e07738439e7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:40.854833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1130/2251] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 10:59:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 10:59:40.858326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b894d1f-3506-46ef-885d-8e07738439e7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:40.864161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c455d9-c178-4a9d-9f2b-fdda9b108f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:40.864853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c455d9-c178-4a9d-9f2b-fdda9b108f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.865140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c455d9-c178-4a9d-9f2b-fdda9b108f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.865802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c455d9-c178-4a9d-9f2b-fdda9b108f44 admin admin] PATH_INFO: `/v3/projects/4136bb33e1b44954a30db420b5cd8afa/users/44841008a38b4964b71c7c21e16db4a6/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.884387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c455d9-c178-4a9d-9f2b-fdda9b108f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4136bb33e1b44954a30db420b5cd8afa, user_id=44841008a38b4964b71c7c21e16db4a6, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:40.890407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c455d9-c178-4a9d-9f2b-fdda9b108f44 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:40.905167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1122/2252] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 10:59:40 2022] PUT /identity/v3/projects/4136bb33e1b44954a30db420b5cd8afa/users/44841008a38b4964b71c7c21e16db4a6/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 10:59:40.915982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56918bc-79bf-4f7b-95bd-4d58be1fbfaa None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:40.916204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56918bc-79bf-4f7b-95bd-4d58be1fbfaa None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:40.916512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56918bc-79bf-4f7b-95bd-4d58be1fbfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:40.940513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:40.949637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-a56918bc-79bf-4f7b-95bd-4d58be1fbfaa None None] MFA Rules not processed for user `44841008a38b4964b71c7c21e16db4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:40.951132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a56918bc-79bf-4f7b-95bd-4d58be1fbfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:41.019088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1131/2253] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 10:59:40 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:41.036705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c455d9-c178-4a9d-9f2b-fdda9b108f44 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:41.038950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a5c455d9-c178-4a9d-9f2b-fdda9b108f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:41.106852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c455d9-c178-4a9d-9f2b-fdda9b108f44 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:41.110411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754b449-4622-4157-8527-28acee2ab6cf None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:41.110788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754b449-4622-4157-8527-28acee2ab6cf None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:41.111063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754b449-4622-4157-8527-28acee2ab6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:41.111876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1df10-86d6-4473-9312-7738dbf22b34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:41.112638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1df10-86d6-4473-9312-7738dbf22b34 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:41.112942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1df10-86d6-4473-9312-7738dbf22b34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:41.113206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1df10-86d6-4473-9312-7738dbf22b34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:41.121872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1df10-86d6-4473-9312-7738dbf22b34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:41.123348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1df10-86d6-4473-9312-7738dbf22b34 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:41.133735 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-8754b449-4622-4157-8527-28acee2ab6cf None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:41.134460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1123/2254] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 10:59:41 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 10:59:41.137277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8754b449-4622-4157-8527-28acee2ab6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:41.204413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1132/2255] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 10:59:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:41.217523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1df10-86d6-4473-9312-7738dbf22b34 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:41.296084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1df10-86d6-4473-9312-7738dbf22b34 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:41.301334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24423f78-012a-454a-8f13-6fd5abc85e41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:41.301937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24423f78-012a-454a-8f13-6fd5abc85e41 service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:41.302216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24423f78-012a-454a-8f13-6fd5abc85e41 service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:41.302482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24423f78-012a-454a-8f13-6fd5abc85e41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:41.310719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24423f78-012a-454a-8f13-6fd5abc85e41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:41.312303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24423f78-012a-454a-8f13-6fd5abc85e41 service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:41.320462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1124/2256] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 10:59:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 10:59:41.678776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11d4ab-9694-40b8-b42a-31e170e70cab None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:41.679093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11d4ab-9694-40b8-b42a-31e170e70cab None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:41.679367 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11d4ab-9694-40b8-b42a-31e170e70cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:41.698928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ff11d4ab-9694-40b8-b42a-31e170e70cab None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 10:59:41.700135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ff11d4ab-9694-40b8-b42a-31e170e70cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 10:59:41.707653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1133/2257] 10.222.0.50 () {58 vars in 967 bytes} [Thu Sep 15 10:59:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 10:59:41.720967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24423f78-012a-454a-8f13-6fd5abc85e41 service swift] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 10:59:41.731830 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24423f78-012a-454a-8f13-6fd5abc85e41 service swift] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 10:59:41.738395 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6be8a7-b6e5-4d3d-94dd-dfe7ee376933 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 10:59:41.739001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6be8a7-b6e5-4d3d-94dd-dfe7ee376933 service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 10:59:41.739284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6be8a7-b6e5-4d3d-94dd-dfe7ee376933 service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 10:59:41.739560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6be8a7-b6e5-4d3d-94dd-dfe7ee376933 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 10:59:41.748178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6be8a7-b6e5-4d3d-94dd-dfe7ee376933 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 10:59:41.749847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6be8a7-b6e5-4d3d-94dd-dfe7ee376933 service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 10:59:41.758360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1125/2258] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 10:59:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 10:59:51.693012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=55 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 10:59:51.754798 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=81,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:00:00.750376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11d4ab-9694-40b8-b42a-31e170e70cab None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:00.752191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ff11d4ab-9694-40b8-b42a-31e170e70cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:00.823950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11d4ab-9694-40b8-b42a-31e170e70cab None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:00.829430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:00.830059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:00.830338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:00.830612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] PATH_INFO: `/v3/users/c0da8726d48b42f0bc70ebcd4b32c887` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:00.831743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0da8726d48b42f0bc70ebcd4b32c887)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:00.833704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:00.889879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0da8726d48b42f0bc70ebcd4b32c887'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:00.890605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0da8726d48b42f0bc70ebcd4b32c887'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:00.899341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0da8726d48b42f0bc70ebcd4b32c887'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:00.899982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0da8726d48b42f0bc70ebcd4b32c887'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:00.907147 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0da8726d48b42f0bc70ebcd4b32c887'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:00.926132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1134/2259] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:00:00 2022] DELETE /identity/v3/users/c0da8726d48b42f0bc70ebcd4b32c887 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:00:00.931260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81d52f-a8a6-46e5-ab4b-75b37f26ddf7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:00.931781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81d52f-a8a6-46e5-ab4b-75b37f26ddf7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:00.932028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81d52f-a8a6-46e5-ab4b-75b37f26ddf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:00.953066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7a81d52f-a8a6-46e5-ab4b-75b37f26ddf7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:00:00.954289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7a81d52f-a8a6-46e5-ab4b-75b37f26ddf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:01.022283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1126/2260] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:00:01.032329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:01.033809 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:01.091723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c51d67-a66f-4f9f-afa0-14c68579d24c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:01.096106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ed87-a1ed-4138-bff2-9d3715a237e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:01.096709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ed87-a1ed-4138-bff2-9d3715a237e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:01.097016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ed87-a1ed-4138-bff2-9d3715a237e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:01.097290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ed87-a1ed-4138-bff2-9d3715a237e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:01.106194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675ed87-a1ed-4138-bff2-9d3715a237e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:01.107627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675ed87-a1ed-4138-bff2-9d3715a237e0 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:01.117142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1135/2261] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:00:01.285957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81d52f-a8a6-46e5-ab4b-75b37f26ddf7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:01.287402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7a81d52f-a8a6-46e5-ab4b-75b37f26ddf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:01.299978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81d52f-a8a6-46e5-ab4b-75b37f26ddf7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:01.304538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:01.305230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:01.305509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:01.305779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] PATH_INFO: `/v3/projects/be8add5c995e456f9323d597079c1623` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:01.306885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be8add5c995e456f9323d597079c1623)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:01.308829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:01.358143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be8add5c995e456f9323d597079c1623', 'request_id': 'req-ce711324-309d-4aa7-9033-c1137730eb49'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:01.364295 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be8add5c995e456f9323d597079c1623', 'request_id': 'req-ce711324-309d-4aa7-9033-c1137730eb49'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:01.371491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be8add5c995e456f9323d597079c1623', 'request_id': 'req-ce711324-309d-4aa7-9033-c1137730eb49'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:01.383313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] The token cache is being invalidate because project be8add5c995e456f9323d597079c1623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:00:01.383599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:01.384235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ce711324-309d-4aa7-9033-c1137730eb49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:01.385752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1127/2262] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:00:01 2022] DELETE /identity/v3/projects/be8add5c995e456f9323d597079c1623 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:00:06.884027 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ed87-a1ed-4138-bff2-9d3715a237e0 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:06.885448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8675ed87-a1ed-4138-bff2-9d3715a237e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:06.898554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ed87-a1ed-4138-bff2-9d3715a237e0 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:06.917211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:00:06.957074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:06.957695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:06.957972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:06.958241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] PATH_INFO: `/v3/users/44841008a38b4964b71c7c21e16db4a6` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:06.959345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44841008a38b4964b71c7c21e16db4a6)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:06.961296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:07.008603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44841008a38b4964b71c7c21e16db4a6'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.009301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44841008a38b4964b71c7c21e16db4a6'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.017406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44841008a38b4964b71c7c21e16db4a6'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.018023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44841008a38b4964b71c7c21e16db4a6'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.025428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44841008a38b4964b71c7c21e16db4a6'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.050070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1136/2263] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:00:06 2022] DELETE /identity/v3/users/44841008a38b4964b71c7c21e16db4a6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:00:07.050185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166f18e-e08b-40a0-86f3-776081f2ed0a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:07.050185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166f18e-e08b-40a0-86f3-776081f2ed0a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:07.050185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166f18e-e08b-40a0-86f3-776081f2ed0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:07.069909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5166f18e-e08b-40a0-86f3-776081f2ed0a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:00:07.071383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5166f18e-e08b-40a0-86f3-776081f2ed0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:07.131894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1128/2264] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:00:07.141959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:07.143445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:07.206686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bb5f8-7fbb-4757-bd1d-a0b50bcc0042 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:07.210363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115d9bf-3797-4d6c-9e7b-34846deded6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:07.210973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115d9bf-3797-4d6c-9e7b-34846deded6e service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:07.211252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115d9bf-3797-4d6c-9e7b-34846deded6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:07.211550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115d9bf-3797-4d6c-9e7b-34846deded6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:07.221697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115d9bf-3797-4d6c-9e7b-34846deded6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:07.223122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115d9bf-3797-4d6c-9e7b-34846deded6e service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:07.232734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1137/2265] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:00:07.501175 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166f18e-e08b-40a0-86f3-776081f2ed0a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:07.503038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5166f18e-e08b-40a0-86f3-776081f2ed0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:07.516276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166f18e-e08b-40a0-86f3-776081f2ed0a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:07.521997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:07.521997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:07.522409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:07.522755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] PATH_INFO: `/v3/projects/4136bb33e1b44954a30db420b5cd8afa` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:07.523958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4136bb33e1b44954a30db420b5cd8afa)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:07.526159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:07.549364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:00:07.581639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4136bb33e1b44954a30db420b5cd8afa', 'request_id': 'req-5c7037c3-1c79-4dc0-9999-5649f94142b4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.587010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4136bb33e1b44954a30db420b5cd8afa', 'request_id': 'req-5c7037c3-1c79-4dc0-9999-5649f94142b4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.594636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4136bb33e1b44954a30db420b5cd8afa', 'request_id': 'req-5c7037c3-1c79-4dc0-9999-5649f94142b4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.605620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] The token cache is being invalidate because project 4136bb33e1b44954a30db420b5cd8afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:00:07.605910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.606538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:07.608086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1129/2266] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:00:07 2022] DELETE /identity/v3/projects/4136bb33e1b44954a30db420b5cd8afa => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:00:17.228359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:00:17.602608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:00:39.233906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fd8a8-7c45-44e9-a14f-b5a30e616a5d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:39.234466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fd8a8-7c45-44e9-a14f-b5a30e616a5d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:39.234466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fd8a8-7c45-44e9-a14f-b5a30e616a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:39.254629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-057fd8a8-7c45-44e9-a14f-b5a30e616a5d None None] MFA Rules not processed for user `f4ecdf8ab7d84d35adb866f94aaf3eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:00:39.255872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-057fd8a8-7c45-44e9-a14f-b5a30e616a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:39.311006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1138/2267] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:00:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:00:42.143377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:42.145665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:42.215062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7037c3-1c79-4dc0-9999-5649f94142b4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:42.220479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:42.221161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:42.221442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:42.225079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] PATH_INFO: `/v3/users/d1e32ef95ff84d01881bd418bad9aaa3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:42.225568 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e32ef95ff84d01881bd418bad9aaa3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:42.227551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:42.284416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e32ef95ff84d01881bd418bad9aaa3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.294812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e32ef95ff84d01881bd418bad9aaa3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.295525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e32ef95ff84d01881bd418bad9aaa3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.303720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e32ef95ff84d01881bd418bad9aaa3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.304353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e32ef95ff84d01881bd418bad9aaa3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.325586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1130/2268] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:00:42 2022] DELETE /identity/v3/users/d1e32ef95ff84d01881bd418bad9aaa3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:00:42.327814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea645c-d528-4b28-bc21-9b650af213ad None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:42.328281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea645c-d528-4b28-bc21-9b650af213ad None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:42.328390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea645c-d528-4b28-bc21-9b650af213ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:42.352043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4fea645c-d528-4b28-bc21-9b650af213ad None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:00:42.353416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4fea645c-d528-4b28-bc21-9b650af213ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:42.428579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1139/2269] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:00:42.435288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:42.436829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:42.495979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa6993-a75f-4c47-a1f2-0ea3a14434a4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:42.499617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380884f0-4279-4759-8875-10de5db4dc5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:42.500227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380884f0-4279-4759-8875-10de5db4dc5c service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:42.500505 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380884f0-4279-4759-8875-10de5db4dc5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:42.500775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380884f0-4279-4759-8875-10de5db4dc5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:42.510588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380884f0-4279-4759-8875-10de5db4dc5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:42.512068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380884f0-4279-4759-8875-10de5db4dc5c service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:42.522492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1131/2270] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:00:42.892089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea645c-d528-4b28-bc21-9b650af213ad None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:42.893601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4fea645c-d528-4b28-bc21-9b650af213ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:42.907824 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea645c-d528-4b28-bc21-9b650af213ad None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:42.913261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:42.913898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:42.914174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:42.914442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] PATH_INFO: `/v3/projects/1c54ca425ddf47a3b6b79f3a6baa6e2d` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:42.915581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c54ca425ddf47a3b6b79f3a6baa6e2d)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:42.918778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:42.966014 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c54ca425ddf47a3b6b79f3a6baa6e2d', 'request_id': 'req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.971398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c54ca425ddf47a3b6b79f3a6baa6e2d', 'request_id': 'req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.978660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c54ca425ddf47a3b6b79f3a6baa6e2d', 'request_id': 'req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.989818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] The token cache is being invalidate because project 1c54ca425ddf47a3b6b79f3a6baa6e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:00:42.990106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.990736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-326f2a11-d080-46ec-97b3-b6f5f0b2a9c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:00:42.992224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1140/2271] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:00:42 2022] DELETE /identity/v3/projects/1c54ca425ddf47a3b6b79f3a6baa6e2d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:00:42.996998 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b91e-4ec2-4046-a3d0-3c4952f49cc7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:42.997286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b91e-4ec2-4046-a3d0-3c4952f49cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:42.997572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b91e-4ec2-4046-a3d0-3c4952f49cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.017307 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d239b91e-4ec2-4046-a3d0-3c4952f49cc7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:00:43.018503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d239b91e-4ec2-4046-a3d0-3c4952f49cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:43.036463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:00:43.080539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1132/2272] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:00:43.089539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664671a6-603e-4052-9781-ebbe65189cca None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:43.089825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664671a6-603e-4052-9781-ebbe65189cca None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:43.090093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664671a6-603e-4052-9781-ebbe65189cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.109628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-664671a6-603e-4052-9781-ebbe65189cca None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:00:43.110814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-664671a6-603e-4052-9781-ebbe65189cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:43.119054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1141/2273] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:00:43.123654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b91e-4ec2-4046-a3d0-3c4952f49cc7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:43.134201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b91e-4ec2-4046-a3d0-3c4952f49cc7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:43.138862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821a1ea-069a-4c04-ad1f-19a20f83090b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:43.139458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821a1ea-069a-4c04-ad1f-19a20f83090b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:43.139732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821a1ea-069a-4c04-ad1f-19a20f83090b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:43.140000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821a1ea-069a-4c04-ad1f-19a20f83090b admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.140631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821a1ea-069a-4c04-ad1f-19a20f83090b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:00:43.140965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821a1ea-069a-4c04-ad1f-19a20f83090b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:43.142561 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821a1ea-069a-4c04-ad1f-19a20f83090b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:43.154827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1133/2274] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:00:43.158959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664671a6-603e-4052-9781-ebbe65189cca None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:43.168921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664671a6-603e-4052-9781-ebbe65189cca None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:43.173489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd8d5-43fb-4b13-a8e5-be21d203b213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:43.174077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd8d5-43fb-4b13-a8e5-be21d203b213 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:43.174354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd8d5-43fb-4b13-a8e5-be21d203b213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:43.174617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd8d5-43fb-4b13-a8e5-be21d203b213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.175246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecd8d5-43fb-4b13-a8e5-be21d203b213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:43.178151 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecd8d5-43fb-4b13-a8e5-be21d203b213 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:43.199879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:00:43.207186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1142/2275] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:00:43 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:00:43.212030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821a1ea-069a-4c04-ad1f-19a20f83090b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:43.222041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821a1ea-069a-4c04-ad1f-19a20f83090b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:43.279105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4367840-01eb-4948-92ca-3cf9e46de91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:43.279716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4367840-01eb-4948-92ca-3cf9e46de91a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:43.279999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4367840-01eb-4948-92ca-3cf9e46de91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:43.280278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4367840-01eb-4948-92ca-3cf9e46de91a admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.280945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4367840-01eb-4948-92ca-3cf9e46de91a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:43.282889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4367840-01eb-4948-92ca-3cf9e46de91a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:43.310284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1134/2276] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:00:43 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:00:43.311392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd8d5-43fb-4b13-a8e5-be21d203b213 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:43.321918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd8d5-43fb-4b13-a8e5-be21d203b213 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:43.326667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be091c-27ee-4b09-80dc-cc696a5b2e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:43.327959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be091c-27ee-4b09-80dc-cc696a5b2e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:43.328520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be091c-27ee-4b09-80dc-cc696a5b2e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:43.328615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be091c-27ee-4b09-80dc-cc696a5b2e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.329376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be091c-27ee-4b09-80dc-cc696a5b2e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:43.331152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be091c-27ee-4b09-80dc-cc696a5b2e66 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:43.342378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1143/2277] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:00:43.346747 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4367840-01eb-4948-92ca-3cf9e46de91a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:43.357648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4367840-01eb-4948-92ca-3cf9e46de91a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:43.362333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c402c-8b20-4d5d-9462-000295f31eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:43.363087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c402c-8b20-4d5d-9462-000295f31eac admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:43.363555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c402c-8b20-4d5d-9462-000295f31eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:43.363859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c402c-8b20-4d5d-9462-000295f31eac admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.364564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8c402c-8b20-4d5d-9462-000295f31eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:43.366301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8c402c-8b20-4d5d-9462-000295f31eac admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:43.377035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1135/2278] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:00:43.380982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be091c-27ee-4b09-80dc-cc696a5b2e66 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:43.391615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be091c-27ee-4b09-80dc-cc696a5b2e66 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:43.397487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81765024-8a07-4ea5-a2c0-581483f3469c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:43.398140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81765024-8a07-4ea5-a2c0-581483f3469c admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:43.399663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81765024-8a07-4ea5-a2c0-581483f3469c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:43.399663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81765024-8a07-4ea5-a2c0-581483f3469c admin admin] PATH_INFO: `/v3/projects/c63dfe497afb46ce864be33c80d38af5/users/477c93978454454993ca2b6c4ed9b240/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.432122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81765024-8a07-4ea5-a2c0-581483f3469c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63dfe497afb46ce864be33c80d38af5, user_id=477c93978454454993ca2b6c4ed9b240, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:43.434836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81765024-8a07-4ea5-a2c0-581483f3469c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:43.448978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1144/2279] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:00:43 2022] PUT /identity/v3/projects/c63dfe497afb46ce864be33c80d38af5/users/477c93978454454993ca2b6c4ed9b240/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:00:43.459554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d294dcd-f46d-4889-adf3-ef02f28f6677 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:43.459842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d294dcd-f46d-4889-adf3-ef02f28f6677 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:43.460116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d294dcd-f46d-4889-adf3-ef02f28f6677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.488425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3d294dcd-f46d-4889-adf3-ef02f28f6677 None None] MFA Rules not processed for user `477c93978454454993ca2b6c4ed9b240`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:00:43.489667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3d294dcd-f46d-4889-adf3-ef02f28f6677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:43.552867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1136/2280] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:00:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:00:43.564099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81765024-8a07-4ea5-a2c0-581483f3469c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:00:43.565851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-81765024-8a07-4ea5-a2c0-581483f3469c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:43.623541 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81765024-8a07-4ea5-a2c0-581483f3469c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:00:43.627272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be65b33-4486-43b5-9114-13f4be879fa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:00:43.627876 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be65b33-4486-43b5-9114-13f4be879fa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:43.628158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be65b33-4486-43b5-9114-13f4be879fa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:43.628439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be65b33-4486-43b5-9114-13f4be879fa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:43.636705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be65b33-4486-43b5-9114-13f4be879fa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:00:43.638164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be65b33-4486-43b5-9114-13f4be879fa4 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:00:43.647747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1145/2281] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:00:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:00:49.104981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394198ba-1728-4660-85fc-6f7ef5fdaf14 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:00:49.105498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394198ba-1728-4660-85fc-6f7ef5fdaf14 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:00:49.105610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394198ba-1728-4660-85fc-6f7ef5fdaf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:00:49.131173 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-394198ba-1728-4660-85fc-6f7ef5fdaf14 None None] MFA Rules not processed for user `e1fdbf0fb6b14090bf38fac073d01814`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:00:49.132451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-394198ba-1728-4660-85fc-6f7ef5fdaf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:00:49.147949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:00:49.218619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1137/2282] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:00:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:00:53.642936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:00:59.205081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=17 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:01:03.678908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be65b33-4486-43b5-9114-13f4be879fa4 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:01:03.680686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0be65b33-4486-43b5-9114-13f4be879fa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:01:03.746902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be65b33-4486-43b5-9114-13f4be879fa4 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:01:03.752291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:01:03.752978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:01:03.753603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:01:03.753603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:01:03.755238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:01:03.832247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:01:03.833740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:01:03.844689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1146/2283] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:01:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:01:06.215208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394198ba-1728-4660-85fc-6f7ef5fdaf14 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:01:06.216706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-394198ba-1728-4660-85fc-6f7ef5fdaf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:01:06.229448 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394198ba-1728-4660-85fc-6f7ef5fdaf14 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:01:06.233347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:01:06.233947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:01:06.234230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:01:06.234504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:01:06.236189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:01:06.250665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:01:06.252149 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:01:06.263530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1138/2284] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:01:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:01:13.839919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=54 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:01:16.258892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:01:18.093940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:01:18.095794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:01:18.109240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae608087-0c2b-4d73-8d5a-c604151bfa68 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:01:18.114622 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7af1e-5d26-4bd5-9547-755f4c77f796 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:01:18.115207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7af1e-5d26-4bd5-9547-755f4c77f796 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:01:18.115482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7af1e-5d26-4bd5-9547-755f4c77f796 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:01:18.115808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7af1e-5d26-4bd5-9547-755f4c77f796 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:01:18.118033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-23c7af1e-5d26-4bd5-9547-755f4c77f796 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:01:18.176861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7af1e-5d26-4bd5-9547-755f4c77f796 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:01:18.178463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7af1e-5d26-4bd5-9547-755f4c77f796 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:01:18.187652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1147/2285] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:01:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:01:18.441672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:01:18.443093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:01:18.454804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52143dc0-7f0e-4b41-ae0b-fd80c9561cac service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:01:18.458337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7a0c2-9aca-47f9-83ed-79e2f0933142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:01:18.458924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7a0c2-9aca-47f9-83ed-79e2f0933142 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:01:18.459202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7a0c2-9aca-47f9-83ed-79e2f0933142 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:01:18.459469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7a0c2-9aca-47f9-83ed-79e2f0933142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:01:18.461762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d3e7a0c2-9aca-47f9-83ed-79e2f0933142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:01:18.518764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7a0c2-9aca-47f9-83ed-79e2f0933142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:01:18.520200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7a0c2-9aca-47f9-83ed-79e2f0933142 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:01:18.529808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1139/2286] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:01:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:01:28.183326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=28 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:01:28.525511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=28 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:01:43.300583 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7af1e-5d26-4bd5-9547-755f4c77f796 service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:01:43.312646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7af1e-5d26-4bd5-9547-755f4c77f796 service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:01:43.318397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:01:43.318993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:01:43.319267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:01:43.319534 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:01:43.322262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:01:43.378683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:01:43.380250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:01:43.390075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1148/2287] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:01:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:01:53.385983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=28 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:02:02.881321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7a0c2-9aca-47f9-83ed-79e2f0933142 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:02.890472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7a0c2-9aca-47f9-83ed-79e2f0933142 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:02.893995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a850f-e757-44a1-ae35-9809f2af1e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:02.894594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a850f-e757-44a1-ae35-9809f2af1e43 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:02.894872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a850f-e757-44a1-ae35-9809f2af1e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:02.895144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a850f-e757-44a1-ae35-9809f2af1e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:02.896892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ff0a850f-e757-44a1-ae35-9809f2af1e43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:02.906692 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a850f-e757-44a1-ae35-9809f2af1e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:02.908113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a850f-e757-44a1-ae35-9809f2af1e43 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:02.916912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1140/2288] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:02:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:02:09.911759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:09.913811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:09.929501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d6c01-d488-4926-8551-6664bd1744a6 service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:09.936287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:09.937004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:09.937286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:09.937555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:09.939284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:09.953120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:09.954818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:09.966792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1149/2289] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:02:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:02:12.912424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:02:13.536059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6431acb-15e8-4f89-9919-53c749750340 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:13.536344 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6431acb-15e8-4f89-9919-53c749750340 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:13.536647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6431acb-15e8-4f89-9919-53c749750340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:13.556679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d6431acb-15e8-4f89-9919-53c749750340 None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:02:13.558166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d6431acb-15e8-4f89-9919-53c749750340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:13.624970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1141/2290] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 11:02:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:02:13.674203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:13.675621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:13.748006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1180-89ab-45f7-841d-a8e59e6630bb service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:13.753161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f93723-92eb-4a39-ac6b-d0d4514f35af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:13.753798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f93723-92eb-4a39-ac6b-d0d4514f35af service swift] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:13.754080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f93723-92eb-4a39-ac6b-d0d4514f35af service swift] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:13.754352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f93723-92eb-4a39-ac6b-d0d4514f35af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:13.763505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f93723-92eb-4a39-ac6b-d0d4514f35af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:13.765215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f93723-92eb-4a39-ac6b-d0d4514f35af service swift] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:13.774139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1150/2291] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 11:02:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 11:02:23.613970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=25 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:02:23.770169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=38 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:02:27.585882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6431acb-15e8-4f89-9919-53c749750340 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:27.595305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6431acb-15e8-4f89-9919-53c749750340 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:27.598875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ced54c-3b35-4155-b57e-29ff15ed5764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:27.599476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ced54c-3b35-4155-b57e-29ff15ed5764 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:27.600390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ced54c-3b35-4155-b57e-29ff15ed5764 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:27.600839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ced54c-3b35-4155-b57e-29ff15ed5764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:27.602550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-96ced54c-3b35-4155-b57e-29ff15ed5764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:27.612306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ced54c-3b35-4155-b57e-29ff15ed5764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:27.613786 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ced54c-3b35-4155-b57e-29ff15ed5764 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:27.622034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1142/2292] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:02:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:02:27.751352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba439d04-f865-448b-bad8-2f23bdafe8e1 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:27.751821 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba439d04-f865-448b-bad8-2f23bdafe8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:27.752039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba439d04-f865-448b-bad8-2f23bdafe8e1 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:27.753651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1151/2293] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 11:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 11:02:27.760939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6354cf1-9be4-40ca-b62e-3041ca09822c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:27.761376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6354cf1-9be4-40ca-b62e-3041ca09822c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:27.761734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6354cf1-9be4-40ca-b62e-3041ca09822c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:27.781700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a6354cf1-9be4-40ca-b62e-3041ca09822c None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:02:27.782923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a6354cf1-9be4-40ca-b62e-3041ca09822c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:27.793822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1143/2294] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 11:02:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:02:27.806434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba439d04-f865-448b-bad8-2f23bdafe8e1 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:27.808156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ba439d04-f865-448b-bad8-2f23bdafe8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:27.873203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba439d04-f865-448b-bad8-2f23bdafe8e1 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:27.876745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177bf69-c62e-4ef1-8b96-62cdae796881 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:27.877394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177bf69-c62e-4ef1-8b96-62cdae796881 service glance] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:27.877676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177bf69-c62e-4ef1-8b96-62cdae796881 service glance] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:27.877948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177bf69-c62e-4ef1-8b96-62cdae796881 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:27.888649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177bf69-c62e-4ef1-8b96-62cdae796881 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:27.890129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177bf69-c62e-4ef1-8b96-62cdae796881 service glance] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:27.901324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1152/2295] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:02:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:02:27.972586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c752afb-ed94-44ca-9c1e-819572548a05 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:27.973092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c752afb-ed94-44ca-9c1e-819572548a05 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:27.973369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c752afb-ed94-44ca-9c1e-819572548a05 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:27.975256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1144/2296] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 11:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 11:02:27.981462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed293943-8198-46e2-9cdc-f2caa00abb4e None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:27.981985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed293943-8198-46e2-9cdc-f2caa00abb4e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:27.982482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed293943-8198-46e2-9cdc-f2caa00abb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:28.019600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ed293943-8198-46e2-9cdc-f2caa00abb4e None None] MFA Rules not processed for user `f5e7b08dc3544e76ac6732ca2aa89bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:02:28.020948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ed293943-8198-46e2-9cdc-f2caa00abb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:28.028669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1153/2297] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 11:02:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:02:28.041829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c752afb-ed94-44ca-9c1e-819572548a05 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:28.055442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c752afb-ed94-44ca-9c1e-819572548a05 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:28.059047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77162462-13cd-40d3-bb04-96f495631f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:28.059700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162462-13cd-40d3-bb04-96f495631f0f service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:28.060101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162462-13cd-40d3-bb04-96f495631f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:28.060393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162462-13cd-40d3-bb04-96f495631f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:28.068589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77162462-13cd-40d3-bb04-96f495631f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:28.070071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77162462-13cd-40d3-bb04-96f495631f0f service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:28.078511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1145/2298] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:02:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:02:28.170832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5380eb1-2257-4c49-b6e8-9e0eb7157176 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:28.171115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5380eb1-2257-4c49-b6e8-9e0eb7157176 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:28.171388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5380eb1-2257-4c49-b6e8-9e0eb7157176 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:28.172759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1154/2299] 10.222.0.50 () {54 vars in 921 bytes} [Thu Sep 15 11:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 11:02:28.177646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e14cb-28b0-446b-9a7e-032d055f1e9a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:28.177939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e14cb-28b0-446b-9a7e-032d055f1e9a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:28.178216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e14cb-28b0-446b-9a7e-032d055f1e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:28.197716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ce9e14cb-28b0-446b-9a7e-032d055f1e9a None None] MFA Rules not processed for user `218a015dd2b34435a76ecc8a763562ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:02:28.198944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ce9e14cb-28b0-446b-9a7e-032d055f1e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:28.207892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1146/2300] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Sep 15 11:02:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:02:28.218530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5380eb1-2257-4c49-b6e8-9e0eb7157176 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:28.230179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5380eb1-2257-4c49-b6e8-9e0eb7157176 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:28.233840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d7765-050a-42ec-97a7-9ba646a49b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:28.234425 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d7765-050a-42ec-97a7-9ba646a49b43 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:28.234705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d7765-050a-42ec-97a7-9ba646a49b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:28.234978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d7765-050a-42ec-97a7-9ba646a49b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:28.245017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d7765-050a-42ec-97a7-9ba646a49b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:28.246445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d7765-050a-42ec-97a7-9ba646a49b43 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:28.257197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1155/2301] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:02:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:02:28.321001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e14cb-28b0-446b-9a7e-032d055f1e9a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:28.322390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ce9e14cb-28b0-446b-9a7e-032d055f1e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:28.390223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e14cb-28b0-446b-9a7e-032d055f1e9a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:28.395353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da4d8f-8dc4-4408-bc9c-34092f861c83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:28.395955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da4d8f-8dc4-4408-bc9c-34092f861c83 service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:28.396238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da4d8f-8dc4-4408-bc9c-34092f861c83 service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:28.397081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da4d8f-8dc4-4408-bc9c-34092f861c83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:28.398852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c7da4d8f-8dc4-4408-bc9c-34092f861c83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:28.410000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da4d8f-8dc4-4408-bc9c-34092f861c83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:28.411576 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da4d8f-8dc4-4408-bc9c-34092f861c83 service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:28.422003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1147/2302] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:02:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:02:38.252488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=35 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:02:38.417805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=50 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:02:50.431895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d7765-050a-42ec-97a7-9ba646a49b43 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:50.443890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d7765-050a-42ec-97a7-9ba646a49b43 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:50.449430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f5966-e14c-43c6-a075-8577e4c63ea5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:50.450009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f5966-e14c-43c6-a075-8577e4c63ea5 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:50.450284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f5966-e14c-43c6-a075-8577e4c63ea5 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:50.450548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f5966-e14c-43c6-a075-8577e4c63ea5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:50.452203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d37f5966-e14c-43c6-a075-8577e4c63ea5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:50.462993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f5966-e14c-43c6-a075-8577e4c63ea5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:50.464566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f5966-e14c-43c6-a075-8577e4c63ea5 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:50.473688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1156/2303] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:02:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:02:53.635832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da4d8f-8dc4-4408-bc9c-34092f861c83 service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:02:53.649752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da4d8f-8dc4-4408-bc9c-34092f861c83 service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:02:53.656257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:02:53.657235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:02:53.657748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:02:53.658443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:02:53.660950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:02:53.689936 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:02:53.691749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:02:53.704543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1148/2304] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:02:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:03:00.469451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=6 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:03:03.699277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:03:18.945662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f5966-e14c-43c6-a075-8577e4c63ea5 service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:03:18.954879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f5966-e14c-43c6-a075-8577e4c63ea5 service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:03:18.962248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880f621-43c4-4875-9910-982421715f8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:03:18.962840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880f621-43c4-4875-9910-982421715f8b service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:03:18.963120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880f621-43c4-4875-9910-982421715f8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:03:18.963391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880f621-43c4-4875-9910-982421715f8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:03:18.965215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6880f621-43c4-4875-9910-982421715f8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:03:19.023434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880f621-43c4-4875-9910-982421715f8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:03:19.024907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880f621-43c4-4875-9910-982421715f8b service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:03:19.033830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1157/2305] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:03:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:03:29.029738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=28 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:05:17.058547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:05:17.059929 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:05:17.153439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6d217-5834-4288-8f97-a7ca60bf038e service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:05:17.158906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdce8be-87b9-4ef5-84c5-8f798b545159 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:05:17.159528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdce8be-87b9-4ef5-84c5-8f798b545159 service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:05:17.159814 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdce8be-87b9-4ef5-84c5-8f798b545159 service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:05:17.160088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdce8be-87b9-4ef5-84c5-8f798b545159 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:05:17.161887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5bdce8be-87b9-4ef5-84c5-8f798b545159 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:05:17.177739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdce8be-87b9-4ef5-84c5-8f798b545159 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:05:17.179367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdce8be-87b9-4ef5-84c5-8f798b545159 service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:05:17.191083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1149/2306] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:05:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:05:27.185357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=37 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:05:43.973721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880f621-43c4-4875-9910-982421715f8b service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:05:44.002302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880f621-43c4-4875-9910-982421715f8b service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:05:44.005851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aae85-1c45-4853-ad8d-252b197d0d36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:05:44.006473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aae85-1c45-4853-ad8d-252b197d0d36 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:05:44.006752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aae85-1c45-4853-ad8d-252b197d0d36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:05:44.007016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aae85-1c45-4853-ad8d-252b197d0d36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:05:44.015300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18aae85-1c45-4853-ad8d-252b197d0d36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:05:44.016717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18aae85-1c45-4853-ad8d-252b197d0d36 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:05:44.025609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1158/2307] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:05:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:05:54.021482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=15 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:06:04.524306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdce8be-87b9-4ef5-84c5-8f798b545159 service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:06:04.543974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdce8be-87b9-4ef5-84c5-8f798b545159 service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:06:04.547632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee9a9c-a15f-4d04-93a4-ce2327a2a47d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:06:04.548225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee9a9c-a15f-4d04-93a4-ce2327a2a47d service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:06:04.548507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee9a9c-a15f-4d04-93a4-ce2327a2a47d service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:06:04.548782 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee9a9c-a15f-4d04-93a4-ce2327a2a47d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:06:04.567070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee9a9c-a15f-4d04-93a4-ce2327a2a47d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:06:04.568521 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee9a9c-a15f-4d04-93a4-ce2327a2a47d service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:06:04.590751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1150/2308] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:06:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:06:14.588183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=17 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:06:18.690369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aae85-1c45-4853-ad8d-252b197d0d36 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:06:18.703322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aae85-1c45-4853-ad8d-252b197d0d36 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:06:18.708823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9dbe3-011f-494f-bae7-a6951d663ca3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:06:18.709426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9dbe3-011f-494f-bae7-a6951d663ca3 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:06:18.709727 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9dbe3-011f-494f-bae7-a6951d663ca3 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:06:18.709988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9dbe3-011f-494f-bae7-a6951d663ca3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:06:18.719111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9dbe3-011f-494f-bae7-a6951d663ca3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:06:18.720975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9dbe3-011f-494f-bae7-a6951d663ca3 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:06:18.749330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1159/2309] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:06:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:06:18.865821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee9a9c-a15f-4d04-93a4-ce2327a2a47d service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:06:18.874971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee9a9c-a15f-4d04-93a4-ce2327a2a47d service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:06:18.878534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c92143-e8b5-46c5-ad4a-23abc7a693d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:06:18.879143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c92143-e8b5-46c5-ad4a-23abc7a693d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:06:18.879425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c92143-e8b5-46c5-ad4a-23abc7a693d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:06:18.879696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c92143-e8b5-46c5-ad4a-23abc7a693d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:06:18.888062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c92143-e8b5-46c5-ad4a-23abc7a693d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:06:18.890226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c92143-e8b5-46c5-ad4a-23abc7a693d8 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:06:18.918014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1151/2310] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:06:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:06:28.746347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=13 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:06:28.915337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=13 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:06:43.550866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9dbe3-011f-494f-bae7-a6951d663ca3 service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:06:43.560710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9dbe3-011f-494f-bae7-a6951d663ca3 service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:06:43.565258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67683-f9f1-43d9-95b0-dd52cb19a52e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:06:43.565823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67683-f9f1-43d9-95b0-dd52cb19a52e service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:06:43.566104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67683-f9f1-43d9-95b0-dd52cb19a52e service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:06:43.566376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67683-f9f1-43d9-95b0-dd52cb19a52e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:06:43.575144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c67683-f9f1-43d9-95b0-dd52cb19a52e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:06:43.576578 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c67683-f9f1-43d9-95b0-dd52cb19a52e service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:06:43.605274 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1160/2311] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:06:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:06:53.602564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=13 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:07:04.797871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c92143-e8b5-46c5-ad4a-23abc7a693d8 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:07:04.807242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c92143-e8b5-46c5-ad4a-23abc7a693d8 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:07:04.810945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf9b9-a0c4-4d20-90ca-bd0ac058470b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:07:04.811555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf9b9-a0c4-4d20-90ca-bd0ac058470b service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:07:04.811839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf9b9-a0c4-4d20-90ca-bd0ac058470b service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:07:04.812379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf9b9-a0c4-4d20-90ca-bd0ac058470b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:07:04.825144 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf9b9-a0c4-4d20-90ca-bd0ac058470b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:07:04.826784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf9b9-a0c4-4d20-90ca-bd0ac058470b service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:07:04.840235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1152/2312] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:07:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:07:14.834921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:07:15.747433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67683-f9f1-43d9-95b0-dd52cb19a52e service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:07:15.756757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67683-f9f1-43d9-95b0-dd52cb19a52e service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:07:15.760361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b9c0d-5f0b-4e14-bd18-6577d61570bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:07:15.761067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b9c0d-5f0b-4e14-bd18-6577d61570bc service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:07:15.761352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b9c0d-5f0b-4e14-bd18-6577d61570bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:07:15.761621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b9c0d-5f0b-4e14-bd18-6577d61570bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:07:15.768677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b9c0d-5f0b-4e14-bd18-6577d61570bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:07:15.770183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b9c0d-5f0b-4e14-bd18-6577d61570bc service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:07:15.778006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1161/2313] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:07:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:07:25.774493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=6 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:07:50.877553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf9b9-a0c4-4d20-90ca-bd0ac058470b service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:07:50.889209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf9b9-a0c4-4d20-90ca-bd0ac058470b service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:07:50.894669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc4c24-55dd-4b4b-bce2-8404047ed23b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:07:50.895263 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc4c24-55dd-4b4b-bce2-8404047ed23b service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:07:50.895542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc4c24-55dd-4b4b-bce2-8404047ed23b service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:07:50.895816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc4c24-55dd-4b4b-bce2-8404047ed23b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:07:50.904071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc4c24-55dd-4b4b-bce2-8404047ed23b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:07:50.905688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc4c24-55dd-4b4b-bce2-8404047ed23b service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:07:50.932986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1153/2314] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:07:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:08:00.931024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=13 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:08:07.134670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b9c0d-5f0b-4e14-bd18-6577d61570bc service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:08:07.163013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b9c0d-5f0b-4e14-bd18-6577d61570bc service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:08:07.169003 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689f2e4-a5d9-4e9f-bce1-8bfd9bfef99b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:08:07.169625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689f2e4-a5d9-4e9f-bce1-8bfd9bfef99b service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:07.170150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689f2e4-a5d9-4e9f-bce1-8bfd9bfef99b service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:07.170601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689f2e4-a5d9-4e9f-bce1-8bfd9bfef99b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:07.189707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689f2e4-a5d9-4e9f-bce1-8bfd9bfef99b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:08:07.191462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689f2e4-a5d9-4e9f-bce1-8bfd9bfef99b service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:08:07.220830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1162/2315] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:08:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:08:17.201787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=6 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:08:19.106080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc4c24-55dd-4b4b-bce2-8404047ed23b service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:08:19.115261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc4c24-55dd-4b4b-bce2-8404047ed23b service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:08:19.119361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09875056-4cd9-48c7-a074-c624c4818c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:08:19.119986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09875056-4cd9-48c7-a074-c624c4818c34 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:19.120249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09875056-4cd9-48c7-a074-c624c4818c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:19.120524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09875056-4cd9-48c7-a074-c624c4818c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:19.128843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09875056-4cd9-48c7-a074-c624c4818c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:08:19.130279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09875056-4cd9-48c7-a074-c624c4818c34 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:08:19.158472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1154/2316] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:08:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:08:29.046823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71b08c-5f5a-4d11-800d-cc7cd03eb917 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:29.047324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71b08c-5f5a-4d11-800d-cc7cd03eb917 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:29.047456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71b08c-5f5a-4d11-800d-cc7cd03eb917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:29.109132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-6c71b08c-5f5a-4d11-800d-cc7cd03eb917 None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:08:29.110352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6c71b08c-5f5a-4d11-800d-cc7cd03eb917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:08:29.127271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1163/2317] 10.222.0.50 () {58 vars in 967 bytes} [Thu Sep 15 11:08:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:08:29.155646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=13 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:08:29.174101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09875056-4cd9-48c7-a074-c624c4818c34 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:08:29.196403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09875056-4cd9-48c7-a074-c624c4818c34 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:08:29.202689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc300860-46a5-4351-87ef-4bb8a8f0d509 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:08:29.203305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc300860-46a5-4351-87ef-4bb8a8f0d509 service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:29.203584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc300860-46a5-4351-87ef-4bb8a8f0d509 service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:29.203971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc300860-46a5-4351-87ef-4bb8a8f0d509 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:29.212042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc300860-46a5-4351-87ef-4bb8a8f0d509 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:08:29.214512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc300860-46a5-4351-87ef-4bb8a8f0d509 service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:08:29.222781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1155/2318] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 11:08:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 11:08:30.450156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7d072-3f7a-46b5-af5d-e41b896f2e95 None None] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:30.450726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7d072-3f7a-46b5-af5d-e41b896f2e95 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:30.450726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7d072-3f7a-46b5-af5d-e41b896f2e95 None None] PATH_INFO: `/` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:30.452567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1164/2319] 10.222.0.50 () {54 vars in 912 bytes} [Thu Sep 15 11:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 11:08:30.458518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af04c1d-1b14-4c5b-b51d-1b3648ec5c46 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:30.459054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af04c1d-1b14-4c5b-b51d-1b3648ec5c46 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:30.459249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af04c1d-1b14-4c5b-b51d-1b3648ec5c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:30.513095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7af04c1d-1b14-4c5b-b51d-1b3648ec5c46 None None] MFA Rules not processed for user `0620b2af88034caa8de99e9b48b05f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:08:30.514423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7af04c1d-1b14-4c5b-b51d-1b3648ec5c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:08:30.523784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1156/2320] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 11:08:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:08:30.551019 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7d072-3f7a-46b5-af5d-e41b896f2e95 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:08:30.566866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7d072-3f7a-46b5-af5d-e41b896f2e95 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:08:30.572415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889d3fa-1c15-48ee-b761-fc65aa4987e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:08:30.573058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889d3fa-1c15-48ee-b761-fc65aa4987e1 service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:30.573337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889d3fa-1c15-48ee-b761-fc65aa4987e1 service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:30.573609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889d3fa-1c15-48ee-b761-fc65aa4987e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:30.582382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889d3fa-1c15-48ee-b761-fc65aa4987e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:08:30.583992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889d3fa-1c15-48ee-b761-fc65aa4987e1 service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:08:30.594629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1165/2321] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:08:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:08:33.032274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43fdfb-f74d-4f21-adf6-74e58880fd10 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:33.032753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43fdfb-f74d-4f21-adf6-74e58880fd10 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:33.032879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43fdfb-f74d-4f21-adf6-74e58880fd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:33.073557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cb43fdfb-f74d-4f21-adf6-74e58880fd10 None None] MFA Rules not processed for user `1eeafdaa1db4436abac229a6f48543a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:08:33.074759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cb43fdfb-f74d-4f21-adf6-74e58880fd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:08:33.081571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1157/2322] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:08:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:08:33.384198 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889d3fa-1c15-48ee-b761-fc65aa4987e1 service placement] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:08:33.394298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889d3fa-1c15-48ee-b761-fc65aa4987e1 service placement] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:08:33.398120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efddff5-b932-43d1-b8a7-cce35eede4ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:08:33.398709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efddff5-b932-43d1-b8a7-cce35eede4ce service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:33.398983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efddff5-b932-43d1-b8a7-cce35eede4ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:33.399246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efddff5-b932-43d1-b8a7-cce35eede4ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:33.401773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0efddff5-b932-43d1-b8a7-cce35eede4ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:08:33.424097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efddff5-b932-43d1-b8a7-cce35eede4ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:08:33.425619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efddff5-b932-43d1-b8a7-cce35eede4ce service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:08:33.434323 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1166/2323] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:08:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:08:34.009190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43fdfb-f74d-4f21-adf6-74e58880fd10 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:08:34.022757 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43fdfb-f74d-4f21-adf6-74e58880fd10 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:08:34.028585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872f2dc-9520-4c38-8814-b02179ce07bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:08:34.029753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f2dc-9520-4c38-8814-b02179ce07bb service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:34.030125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f2dc-9520-4c38-8814-b02179ce07bb service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:34.030510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f2dc-9520-4c38-8814-b02179ce07bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:34.049368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872f2dc-9520-4c38-8814-b02179ce07bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:08:34.049368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872f2dc-9520-4c38-8814-b02179ce07bb service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:08:34.059270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1158/2324] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:08:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:08:41.685312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efddff5-b932-43d1-b8a7-cce35eede4ce service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:08:41.700397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efddff5-b932-43d1-b8a7-cce35eede4ce service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:08:41.705914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf205f5-5c28-46e5-be95-7070cd6066ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:08:41.706513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf205f5-5c28-46e5-be95-7070cd6066ed service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:08:41.706790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf205f5-5c28-46e5-be95-7070cd6066ed service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:08:41.707058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf205f5-5c28-46e5-be95-7070cd6066ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:08:41.717045 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf205f5-5c28-46e5-be95-7070cd6066ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:08:41.718650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf205f5-5c28-46e5-be95-7070cd6066ed service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:08:41.730100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1167/2325] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:08:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:08:44.056002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=40 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:08:51.724859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=45 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:10:23.032926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872f2dc-9520-4c38-8814-b02179ce07bb service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:10:23.043748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872f2dc-9520-4c38-8814-b02179ce07bb service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:10:23.048840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de304a87-f4dc-4a2c-8113-ae8ccdb9740b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:10:23.049441 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de304a87-f4dc-4a2c-8113-ae8ccdb9740b service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:10:23.049724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de304a87-f4dc-4a2c-8113-ae8ccdb9740b service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:10:23.049996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de304a87-f4dc-4a2c-8113-ae8ccdb9740b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:10:23.059098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de304a87-f4dc-4a2c-8113-ae8ccdb9740b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:10:23.060685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de304a87-f4dc-4a2c-8113-ae8ccdb9740b service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:10:23.070678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1159/2326] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:10:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:10:33.066276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:10:44.484625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf205f5-5c28-46e5-be95-7070cd6066ed service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:10:44.486210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9bf205f5-5c28-46e5-be95-7070cd6066ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:10:44.555970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf205f5-5c28-46e5-be95-7070cd6066ed service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:10:44.559511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:10:44.560263 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:10:44.561265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:10:44.561671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:10:44.563489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:10:44.647124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:10:44.648574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:10:44.667022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1168/2327] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:10:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:10:54.664404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=66 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:11:18.770392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de304a87-f4dc-4a2c-8113-ae8ccdb9740b service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:11:18.779651 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de304a87-f4dc-4a2c-8113-ae8ccdb9740b service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:11:18.784064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419c1d-283f-42cc-bf68-2c6b76cbcdad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:11:18.784659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419c1d-283f-42cc-bf68-2c6b76cbcdad service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:11:18.785004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419c1d-283f-42cc-bf68-2c6b76cbcdad service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:11:18.785280 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419c1d-283f-42cc-bf68-2c6b76cbcdad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:11:18.787210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-60419c1d-283f-42cc-bf68-2c6b76cbcdad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:11:18.846080 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60419c1d-283f-42cc-bf68-2c6b76cbcdad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:11:18.847570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60419c1d-283f-42cc-bf68-2c6b76cbcdad service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:11:18.856662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1160/2328] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:11:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:11:19.372471 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:11:19.373969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:11:19.458103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46476399-d972-44ce-ba3d-b6530cec34ec service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:11:19.463691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:11:19.464284 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:11:19.464563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:11:19.464860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:11:19.466644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:11:19.543858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:11:19.545597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:11:19.561406 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1169/2329] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:11:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:11:28.852443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=28 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:11:29.552142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=66 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:11:43.932004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419c1d-283f-42cc-bf68-2c6b76cbcdad service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:11:43.941354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419c1d-283f-42cc-bf68-2c6b76cbcdad service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:11:43.945282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:11:43.945891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:11:43.946175 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:11:43.946454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:11:43.948305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:11:44.018951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:11:44.020391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:11:44.029410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1161/2330] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:11:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:11:54.025200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=33 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:12:18.743830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:12:18.745494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:12:18.804862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06206657-ecfb-485e-bad9-01be048138b6 service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:12:18.808710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:12:18.809349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:12:18.809626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:12:18.809907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:12:18.811745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:12:18.822161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:12:18.823639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:12:18.832951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1170/2331] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:12:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:12:28.828423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=26 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:12:49.531039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:12:49.532585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:12:49.543413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e50706-8bc3-48c6-9769-e8850458e3c3 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:12:49.547006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c225e-4400-42c8-a2f8-6908c8f63a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:12:49.547637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c225e-4400-42c8-a2f8-6908c8f63a49 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:12:49.548481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c225e-4400-42c8-a2f8-6908c8f63a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:12:49.548960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c225e-4400-42c8-a2f8-6908c8f63a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:12:49.611647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c225e-4400-42c8-a2f8-6908c8f63a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:12:49.613151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c225e-4400-42c8-a2f8-6908c8f63a49 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:12:49.622078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1162/2332] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:12:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:12:59.617867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=28 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:13:02.431895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:02.433519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:02.533318 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43742394-d57f-472d-89f3-cd028a28b48f service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:02.538516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:02.539154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:02.539429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:02.539702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] PATH_INFO: `/v3/users/477c93978454454993ca2b6c4ed9b240` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:02.540866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=477c93978454454993ca2b6c4ed9b240)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:02.542872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:02.590048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '477c93978454454993ca2b6c4ed9b240'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:02.590635 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '477c93978454454993ca2b6c4ed9b240'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:02.598104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '477c93978454454993ca2b6c4ed9b240'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:02.598765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '477c93978454454993ca2b6c4ed9b240'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:02.605704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '477c93978454454993ca2b6c4ed9b240'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:02.624714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1171/2333] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:13:02 2022] DELETE /identity/v3/users/477c93978454454993ca2b6c4ed9b240 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:13:02.629199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad68c54-4c9b-40c7-95fd-57f7af33ff82 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:02.629722 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad68c54-4c9b-40c7-95fd-57f7af33ff82 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:02.629941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad68c54-4c9b-40c7-95fd-57f7af33ff82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:02.657439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bad68c54-4c9b-40c7-95fd-57f7af33ff82 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:13:02.658634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bad68c54-4c9b-40c7-95fd-57f7af33ff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:02.716721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1163/2334] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:13:02.727609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:02.729027 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:02.784729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1d797-114e-4732-949f-f7ec3e3222a9 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:02.789171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53edb58b-ff57-4f9e-8acf-4f095353f9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:02.789776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edb58b-ff57-4f9e-8acf-4f095353f9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:02.790058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edb58b-ff57-4f9e-8acf-4f095353f9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:02.790330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edb58b-ff57-4f9e-8acf-4f095353f9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:02.796107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:13:02.800992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53edb58b-ff57-4f9e-8acf-4f095353f9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:02.803230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53edb58b-ff57-4f9e-8acf-4f095353f9f6 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:02.813287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1172/2335] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:13:03.087013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad68c54-4c9b-40c7-95fd-57f7af33ff82 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:03.088516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bad68c54-4c9b-40c7-95fd-57f7af33ff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:03.100469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad68c54-4c9b-40c7-95fd-57f7af33ff82 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:03.105390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:03.106029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.106312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.106586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] PATH_INFO: `/v3/projects/c63dfe497afb46ce864be33c80d38af5` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.107805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c63dfe497afb46ce864be33c80d38af5)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:03.109913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:03.159553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c63dfe497afb46ce864be33c80d38af5', 'request_id': 'req-30bf2c24-6354-44fd-a905-66b90aba5471'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:03.165012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c63dfe497afb46ce864be33c80d38af5', 'request_id': 'req-30bf2c24-6354-44fd-a905-66b90aba5471'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:03.172203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c63dfe497afb46ce864be33c80d38af5', 'request_id': 'req-30bf2c24-6354-44fd-a905-66b90aba5471'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:03.183316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] The token cache is being invalidate because project c63dfe497afb46ce864be33c80d38af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:13:03.183611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:03.184240 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-30bf2c24-6354-44fd-a905-66b90aba5471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:13:03.185753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1164/2336] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:13:03 2022] DELETE /identity/v3/projects/c63dfe497afb46ce864be33c80d38af5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:13:03.192236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8fd67-981c-4847-bb58-b90a1281efda None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.192589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8fd67-981c-4847-bb58-b90a1281efda None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.192976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8fd67-981c-4847-bb58-b90a1281efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.232497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9df8fd67-981c-4847-bb58-b90a1281efda None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:13:03.233745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9df8fd67-981c-4847-bb58-b90a1281efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:03.292271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1173/2337] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:13:03.299933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00e799-72fa-41c5-9458-7e302638641c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.300222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00e799-72fa-41c5-9458-7e302638641c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.300551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00e799-72fa-41c5-9458-7e302638641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.320779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-af00e799-72fa-41c5-9458-7e302638641c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:13:03.322010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-af00e799-72fa-41c5-9458-7e302638641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:03.329759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1165/2338] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:13:03.334676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8fd67-981c-4847-bb58-b90a1281efda None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:03.344444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8fd67-981c-4847-bb58-b90a1281efda None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:03.349161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b7b09-cd0a-44da-b773-3a7483c9104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:03.349761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b7b09-cd0a-44da-b773-3a7483c9104f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.350042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b7b09-cd0a-44da-b773-3a7483c9104f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.350314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b7b09-cd0a-44da-b773-3a7483c9104f admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.350981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b7b09-cd0a-44da-b773-3a7483c9104f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:13:03.351306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b7b09-cd0a-44da-b773-3a7483c9104f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:03.352914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b7b09-cd0a-44da-b773-3a7483c9104f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:03.365865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1174/2339] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:13:03.370109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00e799-72fa-41c5-9458-7e302638641c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:03.379915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00e799-72fa-41c5-9458-7e302638641c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:03.384717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01993b9b-021e-4ead-90e5-e2bf8f7e97f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:03.385349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01993b9b-021e-4ead-90e5-e2bf8f7e97f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.385634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01993b9b-021e-4ead-90e5-e2bf8f7e97f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.385911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01993b9b-021e-4ead-90e5-e2bf8f7e97f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.386543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01993b9b-021e-4ead-90e5-e2bf8f7e97f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:03.388468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01993b9b-021e-4ead-90e5-e2bf8f7e97f8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:03.433203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1166/2340] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:13:03 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:13:03.434666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b7b09-cd0a-44da-b773-3a7483c9104f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:03.450115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b7b09-cd0a-44da-b773-3a7483c9104f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:03.513043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b6556-ec03-4aeb-85d9-29dac97090d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:03.513043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b6556-ec03-4aeb-85d9-29dac97090d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.513043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b6556-ec03-4aeb-85d9-29dac97090d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.513043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b6556-ec03-4aeb-85d9-29dac97090d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.513438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b6556-ec03-4aeb-85d9-29dac97090d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:03.513438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b6556-ec03-4aeb-85d9-29dac97090d0 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:03.533016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1175/2341] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:13:03 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:13:03.537254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01993b9b-021e-4ead-90e5-e2bf8f7e97f8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:03.548063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01993b9b-021e-4ead-90e5-e2bf8f7e97f8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:03.553544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06036099-99d1-43d5-b21b-20facd1d0190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:03.554152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06036099-99d1-43d5-b21b-20facd1d0190 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.554430 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06036099-99d1-43d5-b21b-20facd1d0190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.554696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06036099-99d1-43d5-b21b-20facd1d0190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.555336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06036099-99d1-43d5-b21b-20facd1d0190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:03.556961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06036099-99d1-43d5-b21b-20facd1d0190 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:03.568234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1167/2342] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:13:03.572561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b6556-ec03-4aeb-85d9-29dac97090d0 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:03.582615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b6556-ec03-4aeb-85d9-29dac97090d0 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:03.587345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5559e5f-7315-4018-a3b8-a3f46b5a530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:03.587944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5559e5f-7315-4018-a3b8-a3f46b5a530e admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.588262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5559e5f-7315-4018-a3b8-a3f46b5a530e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.588537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5559e5f-7315-4018-a3b8-a3f46b5a530e admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.589208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5559e5f-7315-4018-a3b8-a3f46b5a530e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:03.590861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5559e5f-7315-4018-a3b8-a3f46b5a530e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:03.601379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1176/2343] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:13:03.605349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06036099-99d1-43d5-b21b-20facd1d0190 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:03.615820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06036099-99d1-43d5-b21b-20facd1d0190 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:03.620549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845c1f6-2ef0-4baa-9cea-daf7a0ba87ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:03.621215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845c1f6-2ef0-4baa-9cea-daf7a0ba87ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.621495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845c1f6-2ef0-4baa-9cea-daf7a0ba87ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.621765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845c1f6-2ef0-4baa-9cea-daf7a0ba87ac admin admin] PATH_INFO: `/v3/projects/1c32818de9364d95af65290e4dd9f658/users/545cbe68d61147d2ad06180fc483ed19/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.636909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845c1f6-2ef0-4baa-9cea-daf7a0ba87ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c32818de9364d95af65290e4dd9f658, user_id=545cbe68d61147d2ad06180fc483ed19, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:03.639575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845c1f6-2ef0-4baa-9cea-daf7a0ba87ac admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:03.652151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1168/2344] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:13:03 2022] PUT /identity/v3/projects/1c32818de9364d95af65290e4dd9f658/users/545cbe68d61147d2ad06180fc483ed19/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:13:03.657097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51732676-b21d-4bf4-8279-5ef60678cc00 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.657385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51732676-b21d-4bf4-8279-5ef60678cc00 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.657670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51732676-b21d-4bf4-8279-5ef60678cc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.677651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-51732676-b21d-4bf4-8279-5ef60678cc00 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:13:03.678831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-51732676-b21d-4bf4-8279-5ef60678cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:03.719090 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:13:03.740437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1177/2345] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:13:03.749981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845c1f6-2ef0-4baa-9cea-daf7a0ba87ac admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:03.751679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8845c1f6-2ef0-4baa-9cea-daf7a0ba87ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:03.758464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:13:03.810600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845c1f6-2ef0-4baa-9cea-daf7a0ba87ac admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:03.814333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcefe6-7859-4666-b266-843ffb7d9298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:03.814924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcefe6-7859-4666-b266-843ffb7d9298 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:03.815318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcefe6-7859-4666-b266-843ffb7d9298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:03.815630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcefe6-7859-4666-b266-843ffb7d9298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:03.824898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcefe6-7859-4666-b266-843ffb7d9298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:03.826316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcefe6-7859-4666-b266-843ffb7d9298 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:03.835660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1169/2346] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:13:05.385659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51732676-b21d-4bf4-8279-5ef60678cc00 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:05.387794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-51732676-b21d-4bf4-8279-5ef60678cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:05.411174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51732676-b21d-4bf4-8279-5ef60678cc00 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:05.416640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8f8ae-7a09-438b-886f-69cd26c8bb6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:05.417779 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8f8ae-7a09-438b-886f-69cd26c8bb6a service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:05.418097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8f8ae-7a09-438b-886f-69cd26c8bb6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:05.418388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8f8ae-7a09-438b-886f-69cd26c8bb6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:05.421712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-1cd8f8ae-7a09-438b-886f-69cd26c8bb6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:05.516613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8f8ae-7a09-438b-886f-69cd26c8bb6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:05.518122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8f8ae-7a09-438b-886f-69cd26c8bb6a service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:05.532528 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1178/2347] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:13:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:13:08.089731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505f895-bf5c-4575-ba5e-fc0ec2c77aa5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:08.090266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505f895-bf5c-4575-ba5e-fc0ec2c77aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:08.090266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505f895-bf5c-4575-ba5e-fc0ec2c77aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:08.125842 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3505f895-bf5c-4575-ba5e-fc0ec2c77aa5 None None] MFA Rules not processed for user `545cbe68d61147d2ad06180fc483ed19`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:13:08.127095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3505f895-bf5c-4575-ba5e-fc0ec2c77aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:08.239538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1170/2348] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:13:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:13:08.243595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8f8ae-7a09-438b-886f-69cd26c8bb6a service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:08.264481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8f8ae-7a09-438b-886f-69cd26c8bb6a service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:08.269332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293bc6a-1c04-448a-b257-0a012088eb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:08.269962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293bc6a-1c04-448a-b257-0a012088eb62 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:08.270965 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293bc6a-1c04-448a-b257-0a012088eb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:08.271271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293bc6a-1c04-448a-b257-0a012088eb62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:08.271948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293bc6a-1c04-448a-b257-0a012088eb62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:08.274056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293bc6a-1c04-448a-b257-0a012088eb62 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:08.306953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1179/2349] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:13:08 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:13:08.311158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505f895-bf5c-4575-ba5e-fc0ec2c77aa5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:08.331385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505f895-bf5c-4575-ba5e-fc0ec2c77aa5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:08.395698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e029a-3d33-4031-8a5a-5713f6c05b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:08.396318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e029a-3d33-4031-8a5a-5713f6c05b9c admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:08.396627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e029a-3d33-4031-8a5a-5713f6c05b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:08.396934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e029a-3d33-4031-8a5a-5713f6c05b9c admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:08.397694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e029a-3d33-4031-8a5a-5713f6c05b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:08.399560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e029a-3d33-4031-8a5a-5713f6c05b9c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:08.429419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1171/2350] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:13:08 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:13:08.433351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293bc6a-1c04-448a-b257-0a012088eb62 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:08.445723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293bc6a-1c04-448a-b257-0a012088eb62 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:08.450849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4953f4-6943-4110-a927-263c9cdf993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:08.451468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4953f4-6943-4110-a927-263c9cdf993b admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:08.451749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4953f4-6943-4110-a927-263c9cdf993b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:08.452022 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4953f4-6943-4110-a927-263c9cdf993b admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:08.452669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4953f4-6943-4110-a927-263c9cdf993b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:08.454392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4953f4-6943-4110-a927-263c9cdf993b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:08.466334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1180/2351] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:13:08.470560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e029a-3d33-4031-8a5a-5713f6c05b9c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:08.481589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e029a-3d33-4031-8a5a-5713f6c05b9c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:08.486225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e16c3-0752-414c-9bdb-7a5eefc77808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:08.486880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e16c3-0752-414c-9bdb-7a5eefc77808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:08.487158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e16c3-0752-414c-9bdb-7a5eefc77808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:08.487428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e16c3-0752-414c-9bdb-7a5eefc77808 admin admin] PATH_INFO: `/v3/projects/f1ceebaf2fcc4eca81e3cde72a227609/users/f5b76d7a4ccf441293131542f96be797/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:08.503334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e16c3-0752-414c-9bdb-7a5eefc77808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ceebaf2fcc4eca81e3cde72a227609, user_id=f5b76d7a4ccf441293131542f96be797, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:08.506117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e16c3-0752-414c-9bdb-7a5eefc77808 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:08.523146 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1172/2352] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:13:08 2022] PUT /identity/v3/projects/f1ceebaf2fcc4eca81e3cde72a227609/users/f5b76d7a4ccf441293131542f96be797/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:13:08.527341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4953f4-6943-4110-a927-263c9cdf993b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:08.538513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4953f4-6943-4110-a927-263c9cdf993b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:08.605853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a3be-6eb1-49bc-8c97-121aca3b2b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:08.605853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a3be-6eb1-49bc-8c97-121aca3b2b2d admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:08.606131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a3be-6eb1-49bc-8c97-121aca3b2b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:08.606301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a3be-6eb1-49bc-8c97-121aca3b2b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:08.606945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1a3be-6eb1-49bc-8c97-121aca3b2b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:08.610503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1a3be-6eb1-49bc-8c97-121aca3b2b2d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:08.623639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1181/2353] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:13:08.630716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e16c3-0752-414c-9bdb-7a5eefc77808 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:08.645376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:13:08.652187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e16c3-0752-414c-9bdb-7a5eefc77808 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:08.657778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b6838-20c4-46cb-9d87-b1a46d25e33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:08.662280 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b6838-20c4-46cb-9d87-b1a46d25e33e admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:08.662432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b6838-20c4-46cb-9d87-b1a46d25e33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:08.662779 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b6838-20c4-46cb-9d87-b1a46d25e33e admin admin] PATH_INFO: `/v3/projects/f1ceebaf2fcc4eca81e3cde72a227609/users/f5b76d7a4ccf441293131542f96be797/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:08.666148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b6838-20c4-46cb-9d87-b1a46d25e33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ceebaf2fcc4eca81e3cde72a227609, user_id=f5b76d7a4ccf441293131542f96be797, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:08.669115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b6838-20c4-46cb-9d87-b1a46d25e33e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:08.682239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1173/2354] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:13:08 2022] PUT /identity/v3/projects/f1ceebaf2fcc4eca81e3cde72a227609/users/f5b76d7a4ccf441293131542f96be797/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:13:13.339023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800e88e-7587-4c7d-a966-ecaadc62611e None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:13.340102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800e88e-7587-4c7d-a966-ecaadc62611e None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:13.340544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800e88e-7587-4c7d-a966-ecaadc62611e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:13.390599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-b800e88e-7587-4c7d-a966-ecaadc62611e None None] MFA Rules not processed for user `f5b76d7a4ccf441293131542f96be797`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:13:13.392005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b800e88e-7587-4c7d-a966-ecaadc62611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:13.415601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:13:13.485262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1182/2355] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:13:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:13:13.490399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b6838-20c4-46cb-9d87-b1a46d25e33e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:13.492845 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dd5b6838-20c4-46cb-9d87-b1a46d25e33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:13.574092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b6838-20c4-46cb-9d87-b1a46d25e33e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:13.579710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4035a-bb5c-46de-aa8b-aa3ddb3e19a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:13.580325 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4035a-bb5c-46de-aa8b-aa3ddb3e19a8 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:13.580609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4035a-bb5c-46de-aa8b-aa3ddb3e19a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:13.580911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4035a-bb5c-46de-aa8b-aa3ddb3e19a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:13.642512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4035a-bb5c-46de-aa8b-aa3ddb3e19a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:13.644767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4035a-bb5c-46de-aa8b-aa3ddb3e19a8 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:13.654771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1174/2356] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:13:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:13:14.557491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800e88e-7587-4c7d-a966-ecaadc62611e None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:14.624437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800e88e-7587-4c7d-a966-ecaadc62611e None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:14.628134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64344622-d7bd-4994-baa2-78b18721d4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:14.628990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64344622-d7bd-4994-baa2-78b18721d4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:14.629412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64344622-d7bd-4994-baa2-78b18721d4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:14.629688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64344622-d7bd-4994-baa2-78b18721d4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:14.631676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-64344622-d7bd-4994-baa2-78b18721d4c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:14.646631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64344622-d7bd-4994-baa2-78b18721d4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:14.648383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64344622-d7bd-4994-baa2-78b18721d4c9 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:14.662237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1183/2357] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:13:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:13:14.822827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a8248-f581-48dc-825f-e27a720ce0e9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:14.823242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a8248-f581-48dc-825f-e27a720ce0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:14.823446 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a8248-f581-48dc-825f-e27a720ce0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:14.864983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b55a8248-f581-48dc-825f-e27a720ce0e9 None None] MFA Rules not processed for user `545cbe68d61147d2ad06180fc483ed19`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:13:14.866151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b55a8248-f581-48dc-825f-e27a720ce0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:14.879071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1175/2358] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:13:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:13:18.301119 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64344622-d7bd-4994-baa2-78b18721d4c9 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:13:18.309683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64344622-d7bd-4994-baa2-78b18721d4c9 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:13:18.313376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:13:18.314752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:13:18.315400 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:13:18.315679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:13:18.317781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:13:18.393713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:13:18.395929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:13:18.405307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1184/2359] 10.222.0.50 () {60 vars in 1396 bytes} [Thu Sep 15 11:13:18 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:13:24.854958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:13:28.401118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=78 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:15:28.008762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a8248-f581-48dc-825f-e27a720ce0e9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:15:28.010436 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b55a8248-f581-48dc-825f-e27a720ce0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:15:28.150792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a8248-f581-48dc-825f-e27a720ce0e9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:15:28.156218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c0a7c-cc31-4716-824c-f7928f6454cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:15:28.156862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c0a7c-cc31-4716-824c-f7928f6454cb service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:15:28.157148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c0a7c-cc31-4716-824c-f7928f6454cb service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:15:28.157508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c0a7c-cc31-4716-824c-f7928f6454cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:15:28.159261 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-226c0a7c-cc31-4716-824c-f7928f6454cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:15:28.171158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c0a7c-cc31-4716-824c-f7928f6454cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:15:28.172771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c0a7c-cc31-4716-824c-f7928f6454cb service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:15:28.183695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1176/2360] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:15:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:15:38.179176 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=58 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:16:02.519060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:16:02.521230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:16:02.600207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9dd8e-08a6-48ee-a8dd-d630cc0c2e3a service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:16:02.606533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e3fa5-142e-4d87-aa1d-2c7d48f8333b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:16:02.607183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e3fa5-142e-4d87-aa1d-2c7d48f8333b service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:16:02.607469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e3fa5-142e-4d87-aa1d-2c7d48f8333b service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:16:02.607749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e3fa5-142e-4d87-aa1d-2c7d48f8333b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:16:02.609494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-794e3fa5-142e-4d87-aa1d-2c7d48f8333b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:16:02.622869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e3fa5-142e-4d87-aa1d-2c7d48f8333b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:16:02.624490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e3fa5-142e-4d87-aa1d-2c7d48f8333b service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:16:02.637209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1185/2361] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:16:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:16:12.631620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=21 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:17:21.762683 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c0a7c-cc31-4716-824c-f7928f6454cb service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:17:21.790577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c0a7c-cc31-4716-824c-f7928f6454cb service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:17:21.794188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c04f71-525e-47c0-bcd0-023ff390e86d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:17:21.794778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c04f71-525e-47c0-bcd0-023ff390e86d service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:17:21.795052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c04f71-525e-47c0-bcd0-023ff390e86d service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:17:21.795324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c04f71-525e-47c0-bcd0-023ff390e86d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:17:21.797029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-12c04f71-525e-47c0-bcd0-023ff390e86d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:17:21.806808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c04f71-525e-47c0-bcd0-023ff390e86d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:17:21.808223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c04f71-525e-47c0-bcd0-023ff390e86d service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:17:21.816045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1177/2362] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:17:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:17:31.812587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=15 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:17:49.748148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e3fa5-142e-4d87-aa1d-2c7d48f8333b service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:17:49.774838 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e3fa5-142e-4d87-aa1d-2c7d48f8333b service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:17:49.780436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:17:49.781061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:17:49.781340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:17:49.781608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:17:49.783295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:17:49.854819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:17:49.856340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:17:49.865301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1186/2363] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:17:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:17:59.861101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=40 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:18:05.792499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c04f71-525e-47c0-bcd0-023ff390e86d service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:18:05.801096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c04f71-525e-47c0-bcd0-023ff390e86d service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:18:05.805310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79acb77-450b-4b39-9d7b-371288a0b64d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:18:05.805902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79acb77-450b-4b39-9d7b-371288a0b64d service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:18:05.806183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79acb77-450b-4b39-9d7b-371288a0b64d service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:18:05.806454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79acb77-450b-4b39-9d7b-371288a0b64d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:18:05.816468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79acb77-450b-4b39-9d7b-371288a0b64d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:18:05.817934 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79acb77-450b-4b39-9d7b-371288a0b64d service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:18:05.830047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1178/2364] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:18:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:18:07.169757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:18:07.169757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:18:07.235441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee5cc-670b-4d5f-9aee-ccdcefecdf33 service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:18:07.239025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abbf71-9e7a-4542-9d6f-acb4f1e67bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:18:07.239652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abbf71-9e7a-4542-9d6f-acb4f1e67bbe service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:18:07.239931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abbf71-9e7a-4542-9d6f-acb4f1e67bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:18:07.240201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abbf71-9e7a-4542-9d6f-acb4f1e67bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:18:07.242500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-63abbf71-9e7a-4542-9d6f-acb4f1e67bbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:18:07.310612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abbf71-9e7a-4542-9d6f-acb4f1e67bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:18:07.312117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abbf71-9e7a-4542-9d6f-acb4f1e67bbe service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:18:07.331503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1187/2365] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:18:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:18:07.374040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79acb77-450b-4b39-9d7b-371288a0b64d service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:18:07.383849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79acb77-450b-4b39-9d7b-371288a0b64d service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:18:07.387398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5043531-6312-407a-b41a-f17ef8a01d1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:18:07.387979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5043531-6312-407a-b41a-f17ef8a01d1e service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:18:07.388253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5043531-6312-407a-b41a-f17ef8a01d1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:18:07.388521 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5043531-6312-407a-b41a-f17ef8a01d1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:18:07.390237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d5043531-6312-407a-b41a-f17ef8a01d1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:18:07.457791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5043531-6312-407a-b41a-f17ef8a01d1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:18:07.459233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5043531-6312-407a-b41a-f17ef8a01d1e service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:18:07.477039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1179/2366] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:18:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:18:07.520711 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abbf71-9e7a-4542-9d6f-acb4f1e67bbe service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:18:07.529293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abbf71-9e7a-4542-9d6f-acb4f1e67bbe service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:18:07.533213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30bc63-4d67-4864-9d03-ccc7815cc2e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:18:07.533818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30bc63-4d67-4864-9d03-ccc7815cc2e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:18:07.534100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30bc63-4d67-4864-9d03-ccc7815cc2e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:18:07.534375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30bc63-4d67-4864-9d03-ccc7815cc2e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:18:07.536087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ab30bc63-4d67-4864-9d03-ccc7815cc2e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:18:07.608179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30bc63-4d67-4864-9d03-ccc7815cc2e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:18:07.609673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30bc63-4d67-4864-9d03-ccc7815cc2e0 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:18:07.629189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1188/2367] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:18:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:18:15.335751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5043531-6312-407a-b41a-f17ef8a01d1e service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:18:15.345265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5043531-6312-407a-b41a-f17ef8a01d1e service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:18:15.348947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93647d77-d17c-4afe-bab4-8646661b82e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:18:15.349466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93647d77-d17c-4afe-bab4-8646661b82e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:18:15.350257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93647d77-d17c-4afe-bab4-8646661b82e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:18:15.350531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93647d77-d17c-4afe-bab4-8646661b82e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:18:15.360489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93647d77-d17c-4afe-bab4-8646661b82e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:18:15.361958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93647d77-d17c-4afe-bab4-8646661b82e1 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:18:15.373391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1180/2368] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:18:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:18:17.626363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:18:18.666133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30bc63-4d67-4864-9d03-ccc7815cc2e0 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:18:18.679211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30bc63-4d67-4864-9d03-ccc7815cc2e0 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:18:18.684822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85b7cb-4279-4680-97c7-1bc9fc3e8530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:18:18.685432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85b7cb-4279-4680-97c7-1bc9fc3e8530 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:18:18.685717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85b7cb-4279-4680-97c7-1bc9fc3e8530 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:18:18.685993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85b7cb-4279-4680-97c7-1bc9fc3e8530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:18:18.694390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85b7cb-4279-4680-97c7-1bc9fc3e8530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:18:18.696271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85b7cb-4279-4680-97c7-1bc9fc3e8530 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:18:18.705784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1189/2369] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:18:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:18:25.368519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=48 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:18:28.701280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=6 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:20:07.990550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93647d77-d17c-4afe-bab4-8646661b82e1 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:07.999666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93647d77-d17c-4afe-bab4-8646661b82e1 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:08.003255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4839364-a0f1-4de0-9569-03d42c786a90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:08.003848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4839364-a0f1-4de0-9569-03d42c786a90 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:08.004127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4839364-a0f1-4de0-9569-03d42c786a90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:08.004396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4839364-a0f1-4de0-9569-03d42c786a90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:08.012928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4839364-a0f1-4de0-9569-03d42c786a90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:08.014461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4839364-a0f1-4de0-9569-03d42c786a90 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:08.024589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1181/2370] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:20:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:20:10.340919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85b7cb-4279-4680-97c7-1bc9fc3e8530 service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:10.352017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85b7cb-4279-4680-97c7-1bc9fc3e8530 service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:10.355753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d99c1-734d-40d9-a8da-fce30e6ec88c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:10.356366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d99c1-734d-40d9-a8da-fce30e6ec88c service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:10.356621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d99c1-734d-40d9-a8da-fce30e6ec88c service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:10.356936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d99c1-734d-40d9-a8da-fce30e6ec88c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:10.418381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d99c1-734d-40d9-a8da-fce30e6ec88c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:10.419826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d99c1-734d-40d9-a8da-fce30e6ec88c service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:10.429431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1190/2371] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:20:13.476938 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4839364-a0f1-4de0-9569-03d42c786a90 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:13.495758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4839364-a0f1-4de0-9569-03d42c786a90 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:13.501197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1b6ea-4622-42ab-8cd7-29cf1b86c96c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:13.501930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1b6ea-4622-42ab-8cd7-29cf1b86c96c service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:13.502298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1b6ea-4622-42ab-8cd7-29cf1b86c96c service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:13.502577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1b6ea-4622-42ab-8cd7-29cf1b86c96c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:13.504499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-30c1b6ea-4622-42ab-8cd7-29cf1b86c96c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:13.517118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1b6ea-4622-42ab-8cd7-29cf1b86c96c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:13.519102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1b6ea-4622-42ab-8cd7-29cf1b86c96c service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:13.541092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1182/2372] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:20:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:20:16.737312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d99c1-734d-40d9-a8da-fce30e6ec88c service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:16.747283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d99c1-734d-40d9-a8da-fce30e6ec88c service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:16.751999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:16.752629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:16.752950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:16.753229 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] PATH_INFO: `/v3/users/545cbe68d61147d2ad06180fc483ed19` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:16.754350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=545cbe68d61147d2ad06180fc483ed19)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:16.756305 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:16.808542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '545cbe68d61147d2ad06180fc483ed19'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:16.809228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '545cbe68d61147d2ad06180fc483ed19'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:16.816320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '545cbe68d61147d2ad06180fc483ed19'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:16.817007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '545cbe68d61147d2ad06180fc483ed19'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:16.824972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '545cbe68d61147d2ad06180fc483ed19'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:16.843632 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1191/2373] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:20:16 2022] DELETE /identity/v3/users/545cbe68d61147d2ad06180fc483ed19 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:20:17.037542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1b6ea-4622-42ab-8cd7-29cf1b86c96c service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:17.038486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1b6ea-4622-42ab-8cd7-29cf1b86c96c service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:17.097293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:17.097918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:17.098199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:17.098474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] PATH_INFO: `/v3/projects/1c32818de9364d95af65290e4dd9f658` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:17.099555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c32818de9364d95af65290e4dd9f658)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:17.101603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:17.145961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c32818de9364d95af65290e4dd9f658', 'request_id': 'req-230c2f61-7360-41a2-86df-8ba2c21257a2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.151795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c32818de9364d95af65290e4dd9f658', 'request_id': 'req-230c2f61-7360-41a2-86df-8ba2c21257a2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.159103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c32818de9364d95af65290e4dd9f658', 'request_id': 'req-230c2f61-7360-41a2-86df-8ba2c21257a2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.170592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] The token cache is being invalidate because project 1c32818de9364d95af65290e4dd9f658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:20:17.170882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.171529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.173030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1183/2374] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:20:17 2022] DELETE /identity/v3/projects/1c32818de9364d95af65290e4dd9f658 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:20:17.177121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:17.178545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:17.191740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9400ada-f2ff-4222-af59-198c6294e9a2 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:17.247958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:17.248584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:17.248894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:17.249165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] PATH_INFO: `/v3/users/f5b76d7a4ccf441293131542f96be797` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:17.250272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b76d7a4ccf441293131542f96be797)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:17.252188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:17.257735 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:20:17.299103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b76d7a4ccf441293131542f96be797'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.299740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b76d7a4ccf441293131542f96be797'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.307183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b76d7a4ccf441293131542f96be797'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.307958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b76d7a4ccf441293131542f96be797'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.314921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aefb9591-89d5-48a3-b95b-dfd95c9a4018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b76d7a4ccf441293131542f96be797'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.333531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1192/2375] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:20:17 2022] DELETE /identity/v3/users/f5b76d7a4ccf441293131542f96be797 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:20:17.503970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:17.505409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:17.525787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c2f61-7360-41a2-86df-8ba2c21257a2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:17.584528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:17.585187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:17.585468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:17.585737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] PATH_INFO: `/v3/projects/f1ceebaf2fcc4eca81e3cde72a227609` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:17.587102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ceebaf2fcc4eca81e3cde72a227609)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:17.589086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:17.595171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:20:17.638682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ceebaf2fcc4eca81e3cde72a227609', 'request_id': 'req-2b495238-f50a-44af-9c49-3e4c59186671'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.644669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ceebaf2fcc4eca81e3cde72a227609', 'request_id': 'req-2b495238-f50a-44af-9c49-3e4c59186671'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.652125 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ceebaf2fcc4eca81e3cde72a227609', 'request_id': 'req-2b495238-f50a-44af-9c49-3e4c59186671'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.663332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] The token cache is being invalidate because project f1ceebaf2fcc4eca81e3cde72a227609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:20:17.663625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.664251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2b495238-f50a-44af-9c49-3e4c59186671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:20:17.665992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1184/2376] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:20:17 2022] DELETE /identity/v3/projects/f1ceebaf2fcc4eca81e3cde72a227609 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:20:17.671051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30912c1-53bc-4699-a420-95f8c6184bd8 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:17.671341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30912c1-53bc-4699-a420-95f8c6184bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:17.671617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30912c1-53bc-4699-a420-95f8c6184bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:17.702494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-a30912c1-53bc-4699-a420-95f8c6184bd8 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:20:17.703620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a30912c1-53bc-4699-a420-95f8c6184bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:17.762533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1193/2377] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:20:17.770731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76cdfb-07cd-4431-b423-7c96dfad16f1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:17.771027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76cdfb-07cd-4431-b423-7c96dfad16f1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:17.771305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76cdfb-07cd-4431-b423-7c96dfad16f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:17.791104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5d76cdfb-07cd-4431-b423-7c96dfad16f1 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:20:17.792318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5d76cdfb-07cd-4431-b423-7c96dfad16f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:17.800235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1185/2378] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:20:17.804507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30912c1-53bc-4699-a420-95f8c6184bd8 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:17.815330 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30912c1-53bc-4699-a420-95f8c6184bd8 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:17.820213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3c146-aa9f-4af2-abf0-44d16620fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:17.821580 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3c146-aa9f-4af2-abf0-44d16620fa3c admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:17.822049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3c146-aa9f-4af2-abf0-44d16620fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:17.822361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3c146-aa9f-4af2-abf0-44d16620fa3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:17.823010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3c146-aa9f-4af2-abf0-44d16620fa3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:20:17.823313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3c146-aa9f-4af2-abf0-44d16620fa3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:17.824954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3c146-aa9f-4af2-abf0-44d16620fa3c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:17.836430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1194/2379] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 11:20:17.840767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76cdfb-07cd-4431-b423-7c96dfad16f1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:17.849966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76cdfb-07cd-4431-b423-7c96dfad16f1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:17.854758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b6d50-6313-4e33-a3b1-439827000c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:17.855357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b6d50-6313-4e33-a3b1-439827000c2d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:17.855643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b6d50-6313-4e33-a3b1-439827000c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:17.855920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b6d50-6313-4e33-a3b1-439827000c2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:17.856579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b6d50-6313-4e33-a3b1-439827000c2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:17.858538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b6d50-6313-4e33-a3b1-439827000c2d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:17.886566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1186/2380] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:20:17 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:20:17.890319 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3c146-aa9f-4af2-abf0-44d16620fa3c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:17.900459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3c146-aa9f-4af2-abf0-44d16620fa3c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:17.957354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a1686-49c7-438d-aec0-67be65a29573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:17.957878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a1686-49c7-438d-aec0-67be65a29573 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:17.958157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a1686-49c7-438d-aec0-67be65a29573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:17.958421 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a1686-49c7-438d-aec0-67be65a29573 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:17.959034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a1686-49c7-438d-aec0-67be65a29573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:17.960975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a1686-49c7-438d-aec0-67be65a29573 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:17.970954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:20:17.984137 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1195/2381] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:20:17 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:20:17.988214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b6d50-6313-4e33-a3b1-439827000c2d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:17.999245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b6d50-6313-4e33-a3b1-439827000c2d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:18.003867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f347c-9bde-4d3f-9329-f49e29521876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:18.004460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f347c-9bde-4d3f-9329-f49e29521876 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:18.004741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f347c-9bde-4d3f-9329-f49e29521876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:18.005046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f347c-9bde-4d3f-9329-f49e29521876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:18.005683 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f347c-9bde-4d3f-9329-f49e29521876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:18.007314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f347c-9bde-4d3f-9329-f49e29521876 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:18.017230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1187/2382] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:20:18.021002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a1686-49c7-438d-aec0-67be65a29573 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:18.031413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a1686-49c7-438d-aec0-67be65a29573 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:18.036082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798fb95-cc2d-4b6f-b8d8-27afe502d6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:18.036673 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798fb95-cc2d-4b6f-b8d8-27afe502d6ce admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:18.036995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798fb95-cc2d-4b6f-b8d8-27afe502d6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:18.037272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798fb95-cc2d-4b6f-b8d8-27afe502d6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:18.037907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798fb95-cc2d-4b6f-b8d8-27afe502d6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:18.040677 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798fb95-cc2d-4b6f-b8d8-27afe502d6ce admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:18.055035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1196/2383] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:20:18.057308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f347c-9bde-4d3f-9329-f49e29521876 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:18.067547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f347c-9bde-4d3f-9329-f49e29521876 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:18.072975 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe0334-8bdf-479e-9014-18aa6af9916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:18.073752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe0334-8bdf-479e-9014-18aa6af9916a admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:18.074152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe0334-8bdf-479e-9014-18aa6af9916a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:18.074508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe0334-8bdf-479e-9014-18aa6af9916a admin admin] PATH_INFO: `/v3/projects/c92e1f7a020b4269bfd9c3e9fdee95a6/users/659d64b97ebf49959e6fbf61f8255ee5/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:18.091289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe0334-8bdf-479e-9014-18aa6af9916a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c92e1f7a020b4269bfd9c3e9fdee95a6, user_id=659d64b97ebf49959e6fbf61f8255ee5, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:18.094118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe0334-8bdf-479e-9014-18aa6af9916a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:18.110974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1188/2384] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:20:18 2022] PUT /identity/v3/projects/c92e1f7a020b4269bfd9c3e9fdee95a6/users/659d64b97ebf49959e6fbf61f8255ee5/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:20:18.116221 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f59e83-1bde-4e5e-90cb-26a2db721047 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:18.116510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f59e83-1bde-4e5e-90cb-26a2db721047 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:18.116782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f59e83-1bde-4e5e-90cb-26a2db721047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:18.137868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-a4f59e83-1bde-4e5e-90cb-26a2db721047 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:20:18.139177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a4f59e83-1bde-4e5e-90cb-26a2db721047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:18.206713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1197/2385] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:20:18.216522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe0334-8bdf-479e-9014-18aa6af9916a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:18.218507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c2fe0334-8bdf-479e-9014-18aa6af9916a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:18.292852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe0334-8bdf-479e-9014-18aa6af9916a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:18.296745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a96f-856a-4641-9542-b80e19b62461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:18.297385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a96f-856a-4641-9542-b80e19b62461 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:18.297671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a96f-856a-4641-9542-b80e19b62461 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:18.297962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a96f-856a-4641-9542-b80e19b62461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:18.309903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a96f-856a-4641-9542-b80e19b62461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:18.311391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a96f-856a-4641-9542-b80e19b62461 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:18.321698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1189/2386] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:20:22.256171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76719-a125-41ab-869d-df938e8f9958 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:22.256644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76719-a125-41ab-869d-df938e8f9958 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:22.256757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76719-a125-41ab-869d-df938e8f9958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:22.288156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-75e76719-a125-41ab-869d-df938e8f9958 None None] MFA Rules not processed for user `659d64b97ebf49959e6fbf61f8255ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:20:22.289810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-75e76719-a125-41ab-869d-df938e8f9958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:22.361860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1198/2387] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:20:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:20:22.366590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a96f-856a-4641-9542-b80e19b62461 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:22.377384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a96f-856a-4641-9542-b80e19b62461 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:22.382189 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e143b36-fd71-4f7c-9435-608b47a1ccc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:22.382797 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e143b36-fd71-4f7c-9435-608b47a1ccc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:22.384027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e143b36-fd71-4f7c-9435-608b47a1ccc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:22.384525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e143b36-fd71-4f7c-9435-608b47a1ccc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:22.385508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e143b36-fd71-4f7c-9435-608b47a1ccc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:22.387679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e143b36-fd71-4f7c-9435-608b47a1ccc8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:22.412181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:20:22.420155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1190/2388] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:20:22 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:20:22.424269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e76719-a125-41ab-869d-df938e8f9958 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:22.438034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e76719-a125-41ab-869d-df938e8f9958 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:22.508870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053856-f955-4ff2-9b97-1edce988e271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:22.513408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053856-f955-4ff2-9b97-1edce988e271 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:22.513408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053856-f955-4ff2-9b97-1edce988e271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:22.513408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053856-f955-4ff2-9b97-1edce988e271 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:22.513408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053856-f955-4ff2-9b97-1edce988e271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:22.519070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053856-f955-4ff2-9b97-1edce988e271 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:22.538361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1199/2389] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:20:22 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:20:22.543083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e143b36-fd71-4f7c-9435-608b47a1ccc8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:22.556443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e143b36-fd71-4f7c-9435-608b47a1ccc8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:22.562023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febf67f-14b9-4fe3-92ec-83cd9f9ffe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:22.562665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febf67f-14b9-4fe3-92ec-83cd9f9ffe83 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:22.562948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febf67f-14b9-4fe3-92ec-83cd9f9ffe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:22.563219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febf67f-14b9-4fe3-92ec-83cd9f9ffe83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:22.563870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febf67f-14b9-4fe3-92ec-83cd9f9ffe83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:22.565557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febf67f-14b9-4fe3-92ec-83cd9f9ffe83 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:22.576852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1191/2390] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:20:22.581085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053856-f955-4ff2-9b97-1edce988e271 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:22.586326 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:20:22.594560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053856-f955-4ff2-9b97-1edce988e271 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:22.601179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ba69d-9d0d-46d1-be71-1b2ccc8a7ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:22.602449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ba69d-9d0d-46d1-be71-1b2ccc8a7ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:22.602723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ba69d-9d0d-46d1-be71-1b2ccc8a7ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:22.602995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ba69d-9d0d-46d1-be71-1b2ccc8a7ecc admin admin] PATH_INFO: `/v3/projects/3bc6544a00c2497aae8adc4feaaaf532/users/2136a18e116d4de0b9150294736918c5/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:22.620993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ba69d-9d0d-46d1-be71-1b2ccc8a7ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc6544a00c2497aae8adc4feaaaf532, user_id=2136a18e116d4de0b9150294736918c5, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:22.623695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ba69d-9d0d-46d1-be71-1b2ccc8a7ecc admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:22.637948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1200/2391] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:20:22 2022] PUT /identity/v3/projects/3bc6544a00c2497aae8adc4feaaaf532/users/2136a18e116d4de0b9150294736918c5/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:20:22.641964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febf67f-14b9-4fe3-92ec-83cd9f9ffe83 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:22.652713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febf67f-14b9-4fe3-92ec-83cd9f9ffe83 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:22.725252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61d10c-2596-4ddf-b935-93fd945a9521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:22.726113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61d10c-2596-4ddf-b935-93fd945a9521 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:22.726469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61d10c-2596-4ddf-b935-93fd945a9521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:22.727378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61d10c-2596-4ddf-b935-93fd945a9521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:22.728225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61d10c-2596-4ddf-b935-93fd945a9521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:22.729978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61d10c-2596-4ddf-b935-93fd945a9521 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:22.742109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1192/2392] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:20:22.746561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ba69d-9d0d-46d1-be71-1b2ccc8a7ecc admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:22.757245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ba69d-9d0d-46d1-be71-1b2ccc8a7ecc admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:22.762320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c04b1-2098-4d56-bd51-06d8fc366946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:22.762988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c04b1-2098-4d56-bd51-06d8fc366946 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:22.763271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c04b1-2098-4d56-bd51-06d8fc366946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:22.763543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c04b1-2098-4d56-bd51-06d8fc366946 admin admin] PATH_INFO: `/v3/projects/3bc6544a00c2497aae8adc4feaaaf532/users/2136a18e116d4de0b9150294736918c5/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:22.765495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c04b1-2098-4d56-bd51-06d8fc366946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc6544a00c2497aae8adc4feaaaf532, user_id=2136a18e116d4de0b9150294736918c5, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:22.768160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c04b1-2098-4d56-bd51-06d8fc366946 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:22.783359 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1201/2393] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:20:22 2022] PUT /identity/v3/projects/3bc6544a00c2497aae8adc4feaaaf532/users/2136a18e116d4de0b9150294736918c5/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:20:27.215186 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a55dee-0199-4a12-8914-99dca399fdd7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:27.215665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a55dee-0199-4a12-8914-99dca399fdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:27.215771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a55dee-0199-4a12-8914-99dca399fdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:27.246207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-09a55dee-0199-4a12-8914-99dca399fdd7 None None] MFA Rules not processed for user `2136a18e116d4de0b9150294736918c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:20:27.247527 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-09a55dee-0199-4a12-8914-99dca399fdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:27.320367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1193/2394] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:20:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:20:27.334567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c04b1-2098-4d56-bd51-06d8fc366946 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:27.337017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-670c04b1-2098-4d56-bd51-06d8fc366946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:27.428702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c04b1-2098-4d56-bd51-06d8fc366946 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:27.434785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73793a5-ec47-49cb-a787-2625b9f46229 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:27.435398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73793a5-ec47-49cb-a787-2625b9f46229 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:27.435753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73793a5-ec47-49cb-a787-2625b9f46229 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:27.436100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73793a5-ec47-49cb-a787-2625b9f46229 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:27.499174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73793a5-ec47-49cb-a787-2625b9f46229 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:27.500810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73793a5-ec47-49cb-a787-2625b9f46229 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:27.510214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1202/2395] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:20:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:20:33.991371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a55dee-0199-4a12-8914-99dca399fdd7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:20:33.992970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-09a55dee-0199-4a12-8914-99dca399fdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:34.061996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a55dee-0199-4a12-8914-99dca399fdd7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:20:34.067276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe4e1a-2507-4d89-9293-e153678673d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:20:34.067883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4e1a-2507-4d89-9293-e153678673d5 service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:34.068162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4e1a-2507-4d89-9293-e153678673d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:34.068434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4e1a-2507-4d89-9293-e153678673d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:34.071088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2afe4e1a-2507-4d89-9293-e153678673d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:34.083154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe4e1a-2507-4d89-9293-e153678673d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:20:34.084754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe4e1a-2507-4d89-9293-e153678673d5 service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:20:34.090210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:20:34.095478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1194/2396] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:20:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:20:37.505867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=70,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:20:59.796310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c92a6-5d9b-4986-907b-c43768a20d05 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:20:59.797008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c92a6-5d9b-4986-907b-c43768a20d05 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:20:59.797008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c92a6-5d9b-4986-907b-c43768a20d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:20:59.835010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ea1c92a6-5d9b-4986-907b-c43768a20d05 None None] MFA Rules not processed for user `e1fdbf0fb6b14090bf38fac073d01814`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:20:59.836217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ea1c92a6-5d9b-4986-907b-c43768a20d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:20:59.900023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1203/2397] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:20:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:21:00.215446 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe4e1a-2507-4d89-9293-e153678673d5 service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:00.268306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe4e1a-2507-4d89-9293-e153678673d5 service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:00.271891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68bed0-9ef1-48f3-8892-9746e192f39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:00.272491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68bed0-9ef1-48f3-8892-9746e192f39f service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:00.272775 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68bed0-9ef1-48f3-8892-9746e192f39f service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:00.273087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68bed0-9ef1-48f3-8892-9746e192f39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:00.275520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3e68bed0-9ef1-48f3-8892-9746e192f39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:21:00.284859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68bed0-9ef1-48f3-8892-9746e192f39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:00.286285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68bed0-9ef1-48f3-8892-9746e192f39f service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:00.294855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1195/2398] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:21:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:21:01.635383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c92a6-5d9b-4986-907b-c43768a20d05 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:01.636786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ea1c92a6-5d9b-4986-907b-c43768a20d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:21:01.651535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c92a6-5d9b-4986-907b-c43768a20d05 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:01.656564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d0fdc-e705-433c-8c3f-0accc328a10a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:01.657202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d0fdc-e705-433c-8c3f-0accc328a10a service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:01.657516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d0fdc-e705-433c-8c3f-0accc328a10a service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:01.657790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d0fdc-e705-433c-8c3f-0accc328a10a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:01.659586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ea2d0fdc-e705-433c-8c3f-0accc328a10a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:21:01.671695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d0fdc-e705-433c-8c3f-0accc328a10a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:01.673302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d0fdc-e705-433c-8c3f-0accc328a10a service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:01.683800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1204/2399] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:21:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:21:07.249544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68bed0-9ef1-48f3-8892-9746e192f39f service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:07.266892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68bed0-9ef1-48f3-8892-9746e192f39f service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:07.275048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733021-9081-47e5-bbfe-56336d886800 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:07.275921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733021-9081-47e5-bbfe-56336d886800 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:07.276280 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733021-9081-47e5-bbfe-56336d886800 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:07.276942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733021-9081-47e5-bbfe-56336d886800 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:07.278715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-85733021-9081-47e5-bbfe-56336d886800 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:21:07.292005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733021-9081-47e5-bbfe-56336d886800 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:07.293729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733021-9081-47e5-bbfe-56336d886800 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:07.315872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1196/2400] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:21:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:21:11.678922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=42 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:21:13.367553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d0fdc-e705-433c-8c3f-0accc328a10a service placement] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:13.382202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d0fdc-e705-433c-8c3f-0accc328a10a service placement] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:13.388621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acf0b3-d083-4a6b-beb5-21212a3d42ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:13.389280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acf0b3-d083-4a6b-beb5-21212a3d42ea service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:13.389577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acf0b3-d083-4a6b-beb5-21212a3d42ea service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:13.389829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acf0b3-d083-4a6b-beb5-21212a3d42ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:13.391640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-66acf0b3-d083-4a6b-beb5-21212a3d42ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:21:13.404962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acf0b3-d083-4a6b-beb5-21212a3d42ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:13.406579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acf0b3-d083-4a6b-beb5-21212a3d42ea service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:13.417771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1205/2401] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:21:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:21:16.792624 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733021-9081-47e5-bbfe-56336d886800 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:16.803758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733021-9081-47e5-bbfe-56336d886800 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:16.809926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771938c-8ffe-4a0a-8e9b-640828fbb9bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:16.810465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771938c-8ffe-4a0a-8e9b-640828fbb9bb service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:16.810748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771938c-8ffe-4a0a-8e9b-640828fbb9bb service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:16.811024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771938c-8ffe-4a0a-8e9b-640828fbb9bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:16.812794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9771938c-8ffe-4a0a-8e9b-640828fbb9bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:21:16.825670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771938c-8ffe-4a0a-8e9b-640828fbb9bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:16.827278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771938c-8ffe-4a0a-8e9b-640828fbb9bb service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:16.838489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1197/2402] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:21:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:21:23.412766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=6 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:21:26.833396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=38 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:21:31.947723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acf0b3-d083-4a6b-beb5-21212a3d42ea service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:31.958913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acf0b3-d083-4a6b-beb5-21212a3d42ea service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:31.964722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9fe45-55d7-426d-ad14-9eaab25f87a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:31.965366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9fe45-55d7-426d-ad14-9eaab25f87a2 service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:31.965643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9fe45-55d7-426d-ad14-9eaab25f87a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:31.966031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9fe45-55d7-426d-ad14-9eaab25f87a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:31.967754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8be9fe45-55d7-426d-ad14-9eaab25f87a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:21:31.979372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9fe45-55d7-426d-ad14-9eaab25f87a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:31.981627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9fe45-55d7-426d-ad14-9eaab25f87a2 service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:31.994449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1206/2403] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:21:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:21:35.005665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771938c-8ffe-4a0a-8e9b-640828fbb9bb service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:35.070926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771938c-8ffe-4a0a-8e9b-640828fbb9bb service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:35.075620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:35.076248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:35.076529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:35.076828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] PATH_INFO: `/v3/users/659d64b97ebf49959e6fbf61f8255ee5` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:35.077946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=659d64b97ebf49959e6fbf61f8255ee5)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:35.079873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:35.126919 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659d64b97ebf49959e6fbf61f8255ee5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.134110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659d64b97ebf49959e6fbf61f8255ee5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.134815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659d64b97ebf49959e6fbf61f8255ee5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.142407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659d64b97ebf49959e6fbf61f8255ee5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.143036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659d64b97ebf49959e6fbf61f8255ee5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.160645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1198/2404] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:21:35 2022] DELETE /identity/v3/users/659d64b97ebf49959e6fbf61f8255ee5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:21:35.328720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9fe45-55d7-426d-ad14-9eaab25f87a2 service placement] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:35.339648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9fe45-55d7-426d-ad14-9eaab25f87a2 service placement] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:35.396928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:35.397597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:35.397879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:35.398158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] PATH_INFO: `/v3/projects/c92e1f7a020b4269bfd9c3e9fdee95a6` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:35.399329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c92e1f7a020b4269bfd9c3e9fdee95a6)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:35.401899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:35.448339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c92e1f7a020b4269bfd9c3e9fdee95a6', 'request_id': 'req-d9a2300f-5a20-433b-acad-40b841e16dc2'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.453885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c92e1f7a020b4269bfd9c3e9fdee95a6', 'request_id': 'req-d9a2300f-5a20-433b-acad-40b841e16dc2'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.462285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c92e1f7a020b4269bfd9c3e9fdee95a6', 'request_id': 'req-d9a2300f-5a20-433b-acad-40b841e16dc2'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.474501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] The token cache is being invalidate because project c92e1f7a020b4269bfd9c3e9fdee95a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:21:35.474789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.475438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.477171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1207/2405] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:21:35 2022] DELETE /identity/v3/projects/c92e1f7a020b4269bfd9c3e9fdee95a6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:21:35.481669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:35.483073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:21:35.496110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921f5cd-8769-499e-9285-1960931fbdcc admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:35.551752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:35.552446 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:35.552672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:35.552983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] PATH_INFO: `/v3/users/2136a18e116d4de0b9150294736918c5` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:35.554117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2136a18e116d4de0b9150294736918c5)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:35.556057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:35.579362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:21:35.604464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2136a18e116d4de0b9150294736918c5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.612079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2136a18e116d4de0b9150294736918c5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.612694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2136a18e116d4de0b9150294736918c5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.619602 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2136a18e116d4de0b9150294736918c5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.621017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2136a18e116d4de0b9150294736918c5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.638720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1199/2406] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:21:35 2022] DELETE /identity/v3/users/2136a18e116d4de0b9150294736918c5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:21:35.810736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:21:35.812234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:21:35.826188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2300f-5a20-433b-acad-40b841e16dc2 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:21:35.883184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:21:35.883819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:21:35.884100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:21:35.884374 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] PATH_INFO: `/v3/projects/3bc6544a00c2497aae8adc4feaaaf532` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:21:35.885507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc6544a00c2497aae8adc4feaaaf532)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:21:35.888107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:21:35.909822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:21:35.935862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc6544a00c2497aae8adc4feaaaf532', 'request_id': 'req-a1d72286-6e09-443a-993e-912f5423cc49'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.941415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc6544a00c2497aae8adc4feaaaf532', 'request_id': 'req-a1d72286-6e09-443a-993e-912f5423cc49'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.949224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc6544a00c2497aae8adc4feaaaf532', 'request_id': 'req-a1d72286-6e09-443a-993e-912f5423cc49'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.960847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] The token cache is being invalidate because project 3bc6544a00c2497aae8adc4feaaaf532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:21:35.961149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.961796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:21:35.963331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1208/2407] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:21:35 2022] DELETE /identity/v3/projects/3bc6544a00c2497aae8adc4feaaaf532 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:21:45.634576 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:21:45.957623 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:22:14.314162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:14.315693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:22:14.378092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28889c6-1c84-49e6-b924-0ac0f0326610 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:14.436554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a559e3-4d01-4359-a730-f2bd9d4c5b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:14.437311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a559e3-4d01-4359-a730-f2bd9d4c5b4c service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:14.437591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a559e3-4d01-4359-a730-f2bd9d4c5b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:14.437862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a559e3-4d01-4359-a730-f2bd9d4c5b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:14.439593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-61a559e3-4d01-4359-a730-f2bd9d4c5b4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:22:14.513282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a559e3-4d01-4359-a730-f2bd9d4c5b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:14.514750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a559e3-4d01-4359-a730-f2bd9d4c5b4c service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:14.524864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1200/2408] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:22:17.824551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:17.826195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:22:17.846059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d72286-6e09-443a-993e-912f5423cc49 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:17.850899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:17.851594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:17.851892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:17.852380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] PATH_INFO: `/v3/users/c4139af75eec4ac6a1576f5a40b02e43` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:17.868592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4139af75eec4ac6a1576f5a40b02e43)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:17.871361 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:17.930368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4139af75eec4ac6a1576f5a40b02e43'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:17.931045 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4139af75eec4ac6a1576f5a40b02e43'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:17.940314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4139af75eec4ac6a1576f5a40b02e43'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:17.940992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4139af75eec4ac6a1576f5a40b02e43'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:17.948988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f2112984-fc3a-4ef3-8371-3e5e559412e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4139af75eec4ac6a1576f5a40b02e43'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:17.970240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1209/2409] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:22:17 2022] DELETE /identity/v3/users/c4139af75eec4ac6a1576f5a40b02e43 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:22:18.156183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a559e3-4d01-4359-a730-f2bd9d4c5b4c service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:18.167438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a559e3-4d01-4359-a730-f2bd9d4c5b4c service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:18.232701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:18.233392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.233671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.233943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] PATH_INFO: `/v3/projects/45cad0c26ac8434da1c3cf7c48aacc39` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.235044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=45cad0c26ac8434da1c3cf7c48aacc39)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:18.237047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:18.246022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:22:18.286631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45cad0c26ac8434da1c3cf7c48aacc39', 'request_id': 'req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:18.292862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45cad0c26ac8434da1c3cf7c48aacc39', 'request_id': 'req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:18.300452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45cad0c26ac8434da1c3cf7c48aacc39', 'request_id': 'req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:18.312489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] The token cache is being invalidate because project 45cad0c26ac8434da1c3cf7c48aacc39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:22:18.312870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:18.313579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-49dbacbd-bd23-44f8-a01a-6d75fd04da9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:22:18.315183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1201/2410] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:22:18 2022] DELETE /identity/v3/projects/45cad0c26ac8434da1c3cf7c48aacc39 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:22:18.321320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06224459-8715-4a06-b560-0164fb21d8ca None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.321750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06224459-8715-4a06-b560-0164fb21d8ca None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.322072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06224459-8715-4a06-b560-0164fb21d8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.344156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-06224459-8715-4a06-b560-0164fb21d8ca None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:22:18.345440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-06224459-8715-4a06-b560-0164fb21d8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:22:18.408634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1210/2411] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:22:18.416557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9930-c4ac-457a-ba96-d407e9c32bb9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.416905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9930-c4ac-457a-ba96-d407e9c32bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.417160 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9930-c4ac-457a-ba96-d407e9c32bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.438600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-134c9930-c4ac-457a-ba96-d407e9c32bb9 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:22:18.439829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-134c9930-c4ac-457a-ba96-d407e9c32bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:22:18.451540 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1202/2412] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:22:18.455886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06224459-8715-4a06-b560-0164fb21d8ca None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:18.465841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06224459-8715-4a06-b560-0164fb21d8ca None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:18.471683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ccddb-9e09-46c4-bcec-13eb525482c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:18.472304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ccddb-9e09-46c4-bcec-13eb525482c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.472605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ccddb-9e09-46c4-bcec-13eb525482c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.472904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ccddb-9e09-46c4-bcec-13eb525482c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.473552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ccddb-9e09-46c4-bcec-13eb525482c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:22:18.473852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ccddb-9e09-46c4-bcec-13eb525482c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:18.475492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ccddb-9e09-46c4-bcec-13eb525482c6 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:18.490293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1211/2413] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:22:18.493486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c9930-c4ac-457a-ba96-d407e9c32bb9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:18.503969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c9930-c4ac-457a-ba96-d407e9c32bb9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:18.508902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43449e-bac9-4366-b5e3-7b01fe6633d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:18.509534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43449e-bac9-4366-b5e3-7b01fe6633d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.509819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43449e-bac9-4366-b5e3-7b01fe6633d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.510094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43449e-bac9-4366-b5e3-7b01fe6633d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.510840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43449e-bac9-4366-b5e3-7b01fe6633d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:18.512890 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43449e-bac9-4366-b5e3-7b01fe6633d5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:18.541361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1203/2414] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:22:18 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:22:18.550107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ccddb-9e09-46c4-bcec-13eb525482c6 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:18.565954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ccddb-9e09-46c4-bcec-13eb525482c6 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:18.613112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:22:18.629601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00047f-74ab-47db-8fb3-77496537617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:18.629780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00047f-74ab-47db-8fb3-77496537617d admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.630064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00047f-74ab-47db-8fb3-77496537617d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.630341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00047f-74ab-47db-8fb3-77496537617d admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.630982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00047f-74ab-47db-8fb3-77496537617d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:18.632993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00047f-74ab-47db-8fb3-77496537617d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:18.654929 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1212/2415] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:22:18 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:22:18.659118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43449e-bac9-4366-b5e3-7b01fe6633d5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:18.670283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43449e-bac9-4366-b5e3-7b01fe6633d5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:18.674931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd96f2-7ead-4c41-8c43-5a1986b5df2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:18.675539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd96f2-7ead-4c41-8c43-5a1986b5df2d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.675819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd96f2-7ead-4c41-8c43-5a1986b5df2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.676092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd96f2-7ead-4c41-8c43-5a1986b5df2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.676733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd96f2-7ead-4c41-8c43-5a1986b5df2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:18.678381 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd96f2-7ead-4c41-8c43-5a1986b5df2d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:18.689254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1204/2416] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:22:18.693704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00047f-74ab-47db-8fb3-77496537617d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:18.704644 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00047f-74ab-47db-8fb3-77496537617d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:18.709758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ca37f-4e59-4990-adb3-6edc9947c31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:18.710356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ca37f-4e59-4990-adb3-6edc9947c31d admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.710632 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ca37f-4e59-4990-adb3-6edc9947c31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.710899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ca37f-4e59-4990-adb3-6edc9947c31d admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.711531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ca37f-4e59-4990-adb3-6edc9947c31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:18.713164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ca37f-4e59-4990-adb3-6edc9947c31d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:18.724851 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1213/2417] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:22:18.730002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd96f2-7ead-4c41-8c43-5a1986b5df2d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:18.739987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd96f2-7ead-4c41-8c43-5a1986b5df2d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:18.744641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf228c-fa7f-4c7d-a719-d55ccce52f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:18.745873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf228c-fa7f-4c7d-a719-d55ccce52f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.746152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf228c-fa7f-4c7d-a719-d55ccce52f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.746428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf228c-fa7f-4c7d-a719-d55ccce52f45 admin admin] PATH_INFO: `/v3/projects/cb988c608283421cbf25e702ad57b87f/users/67f2efb55a1e45d297cf646d4cb87eb2/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.761413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf228c-fa7f-4c7d-a719-d55ccce52f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb988c608283421cbf25e702ad57b87f, user_id=67f2efb55a1e45d297cf646d4cb87eb2, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:18.764109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf228c-fa7f-4c7d-a719-d55ccce52f45 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:18.777274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1205/2418] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:22:18 2022] PUT /identity/v3/projects/cb988c608283421cbf25e702ad57b87f/users/67f2efb55a1e45d297cf646d4cb87eb2/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:22:18.788149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c8a84-6b44-4add-85ab-04ba089fa773 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.788435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c8a84-6b44-4add-85ab-04ba089fa773 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.788707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c8a84-6b44-4add-85ab-04ba089fa773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.817760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-600c8a84-6b44-4add-85ab-04ba089fa773 None None] MFA Rules not processed for user `67f2efb55a1e45d297cf646d4cb87eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:22:18.818894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-600c8a84-6b44-4add-85ab-04ba089fa773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:22:18.884121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1214/2419] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:22:18 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:22:18.889055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf228c-fa7f-4c7d-a719-d55ccce52f45 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:18.950167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf228c-fa7f-4c7d-a719-d55ccce52f45 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:18.954732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cf87d-3d99-4ee7-b294-f8ad63f5f298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:18.955332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cf87d-3d99-4ee7-b294-f8ad63f5f298 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:18.955614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cf87d-3d99-4ee7-b294-f8ad63f5f298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:18.955888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cf87d-3d99-4ee7-b294-f8ad63f5f298 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:18.956520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9cf87d-3d99-4ee7-b294-f8ad63f5f298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:18.958638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9cf87d-3d99-4ee7-b294-f8ad63f5f298 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:18.988368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1206/2420] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:22:18 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:22:18.992464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c8a84-6b44-4add-85ab-04ba089fa773 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:19.003277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c8a84-6b44-4add-85ab-04ba089fa773 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:19.060773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a565a4-2a65-4188-ae79-ae6d440b7b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:19.061446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a565a4-2a65-4188-ae79-ae6d440b7b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:19.061726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a565a4-2a65-4188-ae79-ae6d440b7b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:19.061998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a565a4-2a65-4188-ae79-ae6d440b7b6f admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:19.062669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a565a4-2a65-4188-ae79-ae6d440b7b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:19.064710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a565a4-2a65-4188-ae79-ae6d440b7b6f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:19.084969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1215/2421] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:22:18 2022] POST /identity/v3/users => generated 308 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:22:19.089830 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cf87d-3d99-4ee7-b294-f8ad63f5f298 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:19.099898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cf87d-3d99-4ee7-b294-f8ad63f5f298 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:19.105205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e9ce6-1114-4599-9ef3-d010ef6cd037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:19.105808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e9ce6-1114-4599-9ef3-d010ef6cd037 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:19.106092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e9ce6-1114-4599-9ef3-d010ef6cd037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:19.106369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e9ce6-1114-4599-9ef3-d010ef6cd037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:19.107022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e9ce6-1114-4599-9ef3-d010ef6cd037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:19.108886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e9ce6-1114-4599-9ef3-d010ef6cd037 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:19.121770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1207/2422] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:22:19.125961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a565a4-2a65-4188-ae79-ae6d440b7b6f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:19.136225 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a565a4-2a65-4188-ae79-ae6d440b7b6f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:19.140770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d80606-4c42-4b66-95db-cb1bf5a37a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:19.141451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d80606-4c42-4b66-95db-cb1bf5a37a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:19.141730 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d80606-4c42-4b66-95db-cb1bf5a37a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:19.142004 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d80606-4c42-4b66-95db-cb1bf5a37a8d admin admin] PATH_INFO: `/v3/projects/4e3752091bbe41858b417d17bcd4ead6/users/d1ca890c5d7a48a6b0fc1f7a6b691dfb/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:19.157671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d80606-4c42-4b66-95db-cb1bf5a37a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e3752091bbe41858b417d17bcd4ead6, user_id=d1ca890c5d7a48a6b0fc1f7a6b691dfb, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:19.160413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d80606-4c42-4b66-95db-cb1bf5a37a8d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:19.173607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1216/2423] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:22:19 2022] PUT /identity/v3/projects/4e3752091bbe41858b417d17bcd4ead6/users/d1ca890c5d7a48a6b0fc1f7a6b691dfb/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:22:19.177554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e9ce6-1114-4599-9ef3-d010ef6cd037 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:19.187629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e9ce6-1114-4599-9ef3-d010ef6cd037 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:19.191235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:22:19.246595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a4f72-9e90-44d7-8e0b-e26d4101537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:19.247210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a4f72-9e90-44d7-8e0b-e26d4101537b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:19.247489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a4f72-9e90-44d7-8e0b-e26d4101537b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:19.247762 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a4f72-9e90-44d7-8e0b-e26d4101537b admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:19.248402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a4f72-9e90-44d7-8e0b-e26d4101537b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:19.250202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a4f72-9e90-44d7-8e0b-e26d4101537b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:19.261042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1208/2424] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:22:19.265360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d80606-4c42-4b66-95db-cb1bf5a37a8d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:19.275544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d80606-4c42-4b66-95db-cb1bf5a37a8d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:19.281185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df583c-4c9c-499b-bef4-eb7c048f49c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:19.281840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df583c-4c9c-499b-bef4-eb7c048f49c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:19.282119 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df583c-4c9c-499b-bef4-eb7c048f49c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:19.282392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df583c-4c9c-499b-bef4-eb7c048f49c9 admin admin] PATH_INFO: `/v3/projects/4e3752091bbe41858b417d17bcd4ead6/users/d1ca890c5d7a48a6b0fc1f7a6b691dfb/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:19.284302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df583c-4c9c-499b-bef4-eb7c048f49c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e3752091bbe41858b417d17bcd4ead6, user_id=d1ca890c5d7a48a6b0fc1f7a6b691dfb, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:19.287012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df583c-4c9c-499b-bef4-eb7c048f49c9 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:19.298249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1217/2425] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:22:19 2022] PUT /identity/v3/projects/4e3752091bbe41858b417d17bcd4ead6/users/d1ca890c5d7a48a6b0fc1f7a6b691dfb/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:22:19.310066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c39a0-708d-405f-ad64-df836af44170 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:19.310364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c39a0-708d-405f-ad64-df836af44170 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:19.310632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c39a0-708d-405f-ad64-df836af44170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:19.342299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-479c39a0-708d-405f-ad64-df836af44170 None None] MFA Rules not processed for user `d1ca890c5d7a48a6b0fc1f7a6b691dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:22:19.344748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-479c39a0-708d-405f-ad64-df836af44170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:22:19.414977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1209/2426] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:22:19 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:22:26.498238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df583c-4c9c-499b-bef4-eb7c048f49c9 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:22:26.499959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-59df583c-4c9c-499b-bef4-eb7c048f49c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:22:26.510958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:22:26.559565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df583c-4c9c-499b-bef4-eb7c048f49c9 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:22:26.563182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:22:26.563809 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:22:26.564084 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:22:26.564352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:22:26.566103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:22:26.575978 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:22:26.577445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:22:26.585245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1218/2427] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:22:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:22:29.412207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=60 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:22:36.581843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=22 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:23:07.327980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c39a0-708d-405f-ad64-df836af44170 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:23:07.329514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-479c39a0-708d-405f-ad64-df836af44170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:23:07.341474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c39a0-708d-405f-ad64-df836af44170 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:23:07.344997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50500b25-9d06-404c-ad54-896ea251828f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:23:07.345590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50500b25-9d06-404c-ad54-896ea251828f service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:23:07.345871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50500b25-9d06-404c-ad54-896ea251828f service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:23:07.346141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50500b25-9d06-404c-ad54-896ea251828f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:23:07.347808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-50500b25-9d06-404c-ad54-896ea251828f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:23:07.430816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50500b25-9d06-404c-ad54-896ea251828f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:23:07.432259 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50500b25-9d06-404c-ad54-896ea251828f service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:23:07.443816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1210/2428] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:23:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:23:08.576271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:23:08.577985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:23:08.598823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b6115-c956-4862-97a3-8972207fd918 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:23:08.605386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe20cc-3512-473c-abb8-6e65555fa1ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:23:08.606631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe20cc-3512-473c-abb8-6e65555fa1ee service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:23:08.606631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe20cc-3512-473c-abb8-6e65555fa1ee service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:23:08.606892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe20cc-3512-473c-abb8-6e65555fa1ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:23:08.608533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cffe20cc-3512-473c-abb8-6e65555fa1ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:23:08.693052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe20cc-3512-473c-abb8-6e65555fa1ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:23:08.694580 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe20cc-3512-473c-abb8-6e65555fa1ee service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:23:08.706643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1219/2429] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:23:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:23:16.044303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50500b25-9d06-404c-ad54-896ea251828f service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:23:16.052825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50500b25-9d06-404c-ad54-896ea251828f service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:23:16.056418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41388cdb-008e-4321-9731-476015bd4561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:23:16.057053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41388cdb-008e-4321-9731-476015bd4561 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:23:16.057329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41388cdb-008e-4321-9731-476015bd4561 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:23:16.057601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41388cdb-008e-4321-9731-476015bd4561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:23:16.059497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-41388cdb-008e-4321-9731-476015bd4561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:23:16.081257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41388cdb-008e-4321-9731-476015bd4561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:23:16.082695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41388cdb-008e-4321-9731-476015bd4561 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:23:16.099706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1211/2430] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:23:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:23:18.701745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=33 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:23:26.093939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=45 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:23:32.023757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe20cc-3512-473c-abb8-6e65555fa1ee service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:23:32.035311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe20cc-3512-473c-abb8-6e65555fa1ee service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:23:32.040788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7ea3-f9aa-4f0a-8dc1-68c3c654eddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:23:32.041401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7ea3-f9aa-4f0a-8dc1-68c3c654eddb service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:23:32.041679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7ea3-f9aa-4f0a-8dc1-68c3c654eddb service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:23:32.041951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7ea3-f9aa-4f0a-8dc1-68c3c654eddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:23:32.044567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d48b7ea3-f9aa-4f0a-8dc1-68c3c654eddb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:23:32.103184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b7ea3-f9aa-4f0a-8dc1-68c3c654eddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:23:32.105556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b7ea3-f9aa-4f0a-8dc1-68c3c654eddb service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:23:32.114525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1220/2431] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:23:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:23:37.779760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4cb11-3c02-4233-acb9-4065b8dc5230 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:23:37.780287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4cb11-3c02-4233-acb9-4065b8dc5230 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:23:37.780287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4cb11-3c02-4233-acb9-4065b8dc5230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:23:37.798968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-aea4cb11-3c02-4233-acb9-4065b8dc5230 None None] MFA Rules not processed for user `e1fdbf0fb6b14090bf38fac073d01814`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:23:37.800180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-aea4cb11-3c02-4233-acb9-4065b8dc5230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:23:37.806984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1212/2432] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:23:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:23:38.109596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7ea3-f9aa-4f0a-8dc1-68c3c654eddb service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:23:38.118862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7ea3-f9aa-4f0a-8dc1-68c3c654eddb service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:23:38.123245 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:23:38.123831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:23:38.124130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:23:38.124398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:23:38.126120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:23:38.135455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:23:38.137205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:23:38.145060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1221/2433] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:23:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:23:47.793701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:23:48.141618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=34 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:24:09.347245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4cb11-3c02-4233-acb9-4065b8dc5230 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:24:09.347245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-aea4cb11-3c02-4233-acb9-4065b8dc5230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:24:09.357166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:24:09.359249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:24:09.440967 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4cb11-3c02-4233-acb9-4065b8dc5230 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:24:09.441168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac8636-de69-49bc-915b-3c18e1092f80 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:24:09.444769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8137b-3f37-46e5-8410-757e1ee3ec55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:24:09.445271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:24:09.445653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8137b-3f37-46e5-8410-757e1ee3ec55 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:24:09.446003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8137b-3f37-46e5-8410-757e1ee3ec55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:24:09.446168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:24:09.446414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8137b-3f37-46e5-8410-757e1ee3ec55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:24:09.446521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:24:09.446702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:24:09.449183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-62a8137b-3f37-46e5-8410-757e1ee3ec55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:24:09.449501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:24:09.542136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8137b-3f37-46e5-8410-757e1ee3ec55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:24:09.543611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8137b-3f37-46e5-8410-757e1ee3ec55 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:24:09.555686 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1213/2434] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:24:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:24:09.561461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:24:09.562959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:24:09.584306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1222/2435] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:24:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:24:09.623035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8137b-3f37-46e5-8410-757e1ee3ec55 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:24:09.632791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8137b-3f37-46e5-8410-757e1ee3ec55 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:24:09.637520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22d1b8-0fad-4f3b-b08a-05e476350457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:24:09.638227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22d1b8-0fad-4f3b-b08a-05e476350457 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:24:09.638417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22d1b8-0fad-4f3b-b08a-05e476350457 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:24:09.638690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22d1b8-0fad-4f3b-b08a-05e476350457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:24:09.651514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22d1b8-0fad-4f3b-b08a-05e476350457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:24:09.653034 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22d1b8-0fad-4f3b-b08a-05e476350457 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:24:09.664676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1214/2436] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:24:09 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:24:19.581724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=66 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:24:19.659162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=64 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:25:42.976713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:25:42.978167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:25:43.085181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b0f62-5167-4799-979d-28f0ac5929b1 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:25:43.090253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe365d3-50e6-4a74-999c-2b88a896f151 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:25:43.090850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe365d3-50e6-4a74-999c-2b88a896f151 service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:25:43.091138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe365d3-50e6-4a74-999c-2b88a896f151 service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:25:43.091413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe365d3-50e6-4a74-999c-2b88a896f151 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:25:43.093158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ebe365d3-50e6-4a74-999c-2b88a896f151 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:25:43.104512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe365d3-50e6-4a74-999c-2b88a896f151 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:25:43.106138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe365d3-50e6-4a74-999c-2b88a896f151 service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:25:43.116819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1223/2437] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:25:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:25:53.112926 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=48 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:26:10.345675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22d1b8-0fad-4f3b-b08a-05e476350457 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:10.462859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22d1b8-0fad-4f3b-b08a-05e476350457 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:10.468106 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:10.468749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:10.469073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:10.469348 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] PATH_INFO: `/v3/users/67f2efb55a1e45d297cf646d4cb87eb2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:10.470609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67f2efb55a1e45d297cf646d4cb87eb2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:10.472473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:10.526877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67f2efb55a1e45d297cf646d4cb87eb2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:10.538349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67f2efb55a1e45d297cf646d4cb87eb2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:10.539117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67f2efb55a1e45d297cf646d4cb87eb2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:10.552665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67f2efb55a1e45d297cf646d4cb87eb2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:10.560152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67f2efb55a1e45d297cf646d4cb87eb2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:10.582477 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1215/2438] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:26:10 2022] DELETE /identity/v3/users/67f2efb55a1e45d297cf646d4cb87eb2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:26:10.587692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952bbab-f2ab-4124-ada3-7bd98a977030 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:10.588172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952bbab-f2ab-4124-ada3-7bd98a977030 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:10.588267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952bbab-f2ab-4124-ada3-7bd98a977030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:10.648718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7952bbab-f2ab-4124-ada3-7bd98a977030 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:26:10.650758 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7952bbab-f2ab-4124-ada3-7bd98a977030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:10.719304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1224/2439] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:26:10.729412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:10.730837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:10.778723 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:26:10.792214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884d2bb-d57b-4389-9226-d9968e7c6bf2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:10.796844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68152f-fdeb-4c43-b266-c4ee24152f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:10.797465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68152f-fdeb-4c43-b266-c4ee24152f8d service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:10.797944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68152f-fdeb-4c43-b266-c4ee24152f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:10.798177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68152f-fdeb-4c43-b266-c4ee24152f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:10.808018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68152f-fdeb-4c43-b266-c4ee24152f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:10.809555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68152f-fdeb-4c43-b266-c4ee24152f8d service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:10.823669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1216/2440] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:26:11.118032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952bbab-f2ab-4124-ada3-7bd98a977030 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:11.119529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7952bbab-f2ab-4124-ada3-7bd98a977030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:11.131987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952bbab-f2ab-4124-ada3-7bd98a977030 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:11.137489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:11.138679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:11.138679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:11.138910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] PATH_INFO: `/v3/projects/cb988c608283421cbf25e702ad57b87f` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:11.140130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb988c608283421cbf25e702ad57b87f)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:11.142092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:11.190117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb988c608283421cbf25e702ad57b87f', 'request_id': 'req-aaf275e6-858a-417a-96bc-4144e22d0f09'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.195585 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb988c608283421cbf25e702ad57b87f', 'request_id': 'req-aaf275e6-858a-417a-96bc-4144e22d0f09'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.202812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb988c608283421cbf25e702ad57b87f', 'request_id': 'req-aaf275e6-858a-417a-96bc-4144e22d0f09'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.213818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] The token cache is being invalidate because project cb988c608283421cbf25e702ad57b87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:26:11.214110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.214745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.216742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1225/2441] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:26:11 2022] DELETE /identity/v3/projects/cb988c608283421cbf25e702ad57b87f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:26:11.220690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68152f-fdeb-4c43-b266-c4ee24152f8d service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:11.230611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68152f-fdeb-4c43-b266-c4ee24152f8d service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:11.303056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:11.303857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:11.304243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:11.304672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] PATH_INFO: `/v3/users/d1ca890c5d7a48a6b0fc1f7a6b691dfb` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:11.307324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ca890c5d7a48a6b0fc1f7a6b691dfb)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:11.309499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:11.366924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ca890c5d7a48a6b0fc1f7a6b691dfb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.375112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ca890c5d7a48a6b0fc1f7a6b691dfb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.375839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ca890c5d7a48a6b0fc1f7a6b691dfb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.382983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ca890c5d7a48a6b0fc1f7a6b691dfb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.383674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cbc0d5c7-f9e4-4892-b268-df5160a1b357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ca890c5d7a48a6b0fc1f7a6b691dfb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:11.401609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1217/2442] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:26:11 2022] DELETE /identity/v3/users/d1ca890c5d7a48a6b0fc1f7a6b691dfb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:26:11.872231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:11.874780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:11.901983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf275e6-858a-417a-96bc-4144e22d0f09 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:11.959965 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:26:11.967909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:11.969203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:11.970023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:11.970444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] PATH_INFO: `/v3/projects/4e3752091bbe41858b417d17bcd4ead6` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:11.971859 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e3752091bbe41858b417d17bcd4ead6)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:11.974477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.030204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e3752091bbe41858b417d17bcd4ead6', 'request_id': 'req-ee74a94b-1cee-4687-8262-41dc3ee26a0c'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:12.035975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e3752091bbe41858b417d17bcd4ead6', 'request_id': 'req-ee74a94b-1cee-4687-8262-41dc3ee26a0c'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:12.043840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e3752091bbe41858b417d17bcd4ead6', 'request_id': 'req-ee74a94b-1cee-4687-8262-41dc3ee26a0c'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:12.056269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] The token cache is being invalidate because project 4e3752091bbe41858b417d17bcd4ead6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:26:12.056563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:12.057242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ee74a94b-1cee-4687-8262-41dc3ee26a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:26:12.058754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1226/2443] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:26:11 2022] DELETE /identity/v3/projects/4e3752091bbe41858b417d17bcd4ead6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:26:12.064193 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a8f47-c3f1-4d7d-9c72-fc1307f386f4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.064635 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a8f47-c3f1-4d7d-9c72-fc1307f386f4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.064939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a8f47-c3f1-4d7d-9c72-fc1307f386f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.109181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-347a8f47-c3f1-4d7d-9c72-fc1307f386f4 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:26:12.110449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-347a8f47-c3f1-4d7d-9c72-fc1307f386f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:12.162755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:26:12.184829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1218/2444] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:26:12.193250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844a6fd-ca92-4f8a-8e62-f1e926e5c83c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.193556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844a6fd-ca92-4f8a-8e62-f1e926e5c83c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.193837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844a6fd-ca92-4f8a-8e62-f1e926e5c83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.215437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9844a6fd-ca92-4f8a-8e62-f1e926e5c83c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:26:12.217808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9844a6fd-ca92-4f8a-8e62-f1e926e5c83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:12.230262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1227/2445] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:26:12.231674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a8f47-c3f1-4d7d-9c72-fc1307f386f4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.242893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a8f47-c3f1-4d7d-9c72-fc1307f386f4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.247869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.248536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.248857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.249137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.249793 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:26:12.250096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.251736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.265067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1219/2446] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:26:12.269223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844a6fd-ca92-4f8a-8e62-f1e926e5c83c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.280168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844a6fd-ca92-4f8a-8e62-f1e926e5c83c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.285730 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697c1bf-2bc6-4cb0-a8f5-a3eec66738e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.286395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697c1bf-2bc6-4cb0-a8f5-a3eec66738e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.286760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697c1bf-2bc6-4cb0-a8f5-a3eec66738e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.287276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697c1bf-2bc6-4cb0-a8f5-a3eec66738e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.288157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697c1bf-2bc6-4cb0-a8f5-a3eec66738e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.290315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697c1bf-2bc6-4cb0-a8f5-a3eec66738e3 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.302607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.304504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:12.369936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1228/2447] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:26:12 2022] POST /identity/v3/projects => generated 357 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:26:12.373689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697c1bf-2bc6-4cb0-a8f5-a3eec66738e3 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.427663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db3973-2143-4099-a1de-9ba5e2820614 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.463447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697c1bf-2bc6-4cb0-a8f5-a3eec66738e3 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.470234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dafed1-686b-4ddd-a303-e2bd1d684291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.470234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dafed1-686b-4ddd-a303-e2bd1d684291 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.470988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dafed1-686b-4ddd-a303-e2bd1d684291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.471475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dafed1-686b-4ddd-a303-e2bd1d684291 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.472449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dafed1-686b-4ddd-a303-e2bd1d684291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.474808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dafed1-686b-4ddd-a303-e2bd1d684291 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.503684 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1229/2448] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:26:12 2022] POST /identity/v3/users => generated 279 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:26:12.508630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dafed1-686b-4ddd-a303-e2bd1d684291 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.519172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a390774-5415-408d-9ba0-71ebeab4a2a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.519877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a390774-5415-408d-9ba0-71ebeab4a2a2 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.520277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a390774-5415-408d-9ba0-71ebeab4a2a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.520739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a390774-5415-408d-9ba0-71ebeab4a2a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.521456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dafed1-686b-4ddd-a303-e2bd1d684291 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.523431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7a390774-5415-408d-9ba0-71ebeab4a2a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:12.527351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3bd9-f99b-4e6e-9320-a922cec4cb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.528399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3bd9-f99b-4e6e-9320-a922cec4cb48 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.528917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3bd9-f99b-4e6e-9320-a922cec4cb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.529365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3bd9-f99b-4e6e-9320-a922cec4cb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.530185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f3bd9-f99b-4e6e-9320-a922cec4cb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.532134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f3bd9-f99b-4e6e-9320-a922cec4cb48 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.537556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a390774-5415-408d-9ba0-71ebeab4a2a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.539174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a390774-5415-408d-9ba0-71ebeab4a2a2 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.547565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1230/2449] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:26:12.551641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3bd9-f99b-4e6e-9320-a922cec4cb48 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.552409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1220/2450] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:26:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:26:12.565884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3bd9-f99b-4e6e-9320-a922cec4cb48 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.573625 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb29b93-407a-4aaa-9265-34211cc444ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.574249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb29b93-407a-4aaa-9265-34211cc444ef admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.574538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb29b93-407a-4aaa-9265-34211cc444ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.574805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb29b93-407a-4aaa-9265-34211cc444ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.576050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb29b93-407a-4aaa-9265-34211cc444ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.577811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb29b93-407a-4aaa-9265-34211cc444ef admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.589719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1231/2451] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:26:12.594270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a390774-5415-408d-9ba0-71ebeab4a2a2 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.607288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a390774-5415-408d-9ba0-71ebeab4a2a2 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.612557 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf4a1d-d0da-4d39-8dc7-ac4a03070201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.613215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf4a1d-d0da-4d39-8dc7-ac4a03070201 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.613501 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf4a1d-d0da-4d39-8dc7-ac4a03070201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.613806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf4a1d-d0da-4d39-8dc7-ac4a03070201 admin admin] PATH_INFO: `/v3/projects/f934f93d8bc644de976b3fd821047fa2/users/38945899ee7e4f91af7fee8836cc2710/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.630992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf4a1d-d0da-4d39-8dc7-ac4a03070201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f934f93d8bc644de976b3fd821047fa2, user_id=38945899ee7e4f91af7fee8836cc2710, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.633764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf4a1d-d0da-4d39-8dc7-ac4a03070201 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.647279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1221/2452] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:26:12 2022] PUT /identity/v3/projects/f934f93d8bc644de976b3fd821047fa2/users/38945899ee7e4f91af7fee8836cc2710/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:26:12.658407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403e47a-b448-44e2-8117-3e8db3c3649c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.658588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403e47a-b448-44e2-8117-3e8db3c3649c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.658828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403e47a-b448-44e2-8117-3e8db3c3649c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.687533 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-f403e47a-b448-44e2-8117-3e8db3c3649c None None] MFA Rules not processed for user `38945899ee7e4f91af7fee8836cc2710`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:26:12.689072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f403e47a-b448-44e2-8117-3e8db3c3649c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:12.694156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:26:12.757200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1232/2453] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:26:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:26:12.761804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf4a1d-d0da-4d39-8dc7-ac4a03070201 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.802015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:26:12.827471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf4a1d-d0da-4d39-8dc7-ac4a03070201 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.832260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e3fd6-d626-4cb6-8a1b-793cb5470653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.832917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e3fd6-d626-4cb6-8a1b-793cb5470653 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.833204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e3fd6-d626-4cb6-8a1b-793cb5470653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.833481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e3fd6-d626-4cb6-8a1b-793cb5470653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.834128 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e3fd6-d626-4cb6-8a1b-793cb5470653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.836050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e3fd6-d626-4cb6-8a1b-793cb5470653 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.865607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1222/2454] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:26:12 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:26:12.868584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403e47a-b448-44e2-8117-3e8db3c3649c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.880596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403e47a-b448-44e2-8117-3e8db3c3649c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.943026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd113954-a9a7-451e-902c-8a0900011da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.943706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd113954-a9a7-451e-902c-8a0900011da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.943990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd113954-a9a7-451e-902c-8a0900011da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.944319 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd113954-a9a7-451e-902c-8a0900011da8 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.945241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd113954-a9a7-451e-902c-8a0900011da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.946844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd113954-a9a7-451e-902c-8a0900011da8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:12.973308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1233/2455] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:26:12 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:26:12.975323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e3fd6-d626-4cb6-8a1b-793cb5470653 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:12.986509 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e3fd6-d626-4cb6-8a1b-793cb5470653 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:12.991326 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de430d-41a3-4030-aefc-93c857c8964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:12.992093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de430d-41a3-4030-aefc-93c857c8964a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:12.992399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de430d-41a3-4030-aefc-93c857c8964a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:12.992840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de430d-41a3-4030-aefc-93c857c8964a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:12.993565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de430d-41a3-4030-aefc-93c857c8964a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:12.995357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de430d-41a3-4030-aefc-93c857c8964a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:13.013690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd113954-a9a7-451e-902c-8a0900011da8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:13.018608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1223/2456] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:26:13.024916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd113954-a9a7-451e-902c-8a0900011da8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:13.029971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3006f-4763-4444-8e38-3a465181dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:13.030788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3006f-4763-4444-8e38-3a465181dea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:13.031143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3006f-4763-4444-8e38-3a465181dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:13.031506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3006f-4763-4444-8e38-3a465181dea8 admin admin] PATH_INFO: `/v3/projects/08d221c8992040aaaa2a2ecce4f55e79/users/fa24fe2ccdba4661813bcb962fd9ea5f/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:13.048144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3006f-4763-4444-8e38-3a465181dea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d221c8992040aaaa2a2ecce4f55e79, user_id=fa24fe2ccdba4661813bcb962fd9ea5f, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:13.050969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3006f-4763-4444-8e38-3a465181dea8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:13.065939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1234/2457] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:26:13 2022] PUT /identity/v3/projects/08d221c8992040aaaa2a2ecce4f55e79/users/fa24fe2ccdba4661813bcb962fd9ea5f/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:26:13.069816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de430d-41a3-4030-aefc-93c857c8964a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:13.080360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de430d-41a3-4030-aefc-93c857c8964a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:13.137898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466b3c8-a834-430e-8dea-78111581e3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:13.138520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466b3c8-a834-430e-8dea-78111581e3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:13.138806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466b3c8-a834-430e-8dea-78111581e3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:13.139104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466b3c8-a834-430e-8dea-78111581e3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:13.139738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466b3c8-a834-430e-8dea-78111581e3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:13.141423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466b3c8-a834-430e-8dea-78111581e3ba admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:13.152137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1224/2458] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:26:13.156532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3006f-4763-4444-8e38-3a465181dea8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:26:13.167119 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3006f-4763-4444-8e38-3a465181dea8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:26:13.172819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad7168-910d-4850-ae6c-5a1a8b2b8bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:26:13.173502 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad7168-910d-4850-ae6c-5a1a8b2b8bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:13.173809 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad7168-910d-4850-ae6c-5a1a8b2b8bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:13.174034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad7168-910d-4850-ae6c-5a1a8b2b8bdb admin admin] PATH_INFO: `/v3/projects/08d221c8992040aaaa2a2ecce4f55e79/users/fa24fe2ccdba4661813bcb962fd9ea5f/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:13.175974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad7168-910d-4850-ae6c-5a1a8b2b8bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d221c8992040aaaa2a2ecce4f55e79, user_id=fa24fe2ccdba4661813bcb962fd9ea5f, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:26:13.178691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad7168-910d-4850-ae6c-5a1a8b2b8bdb admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:26:13.192157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1235/2459] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:26:13 2022] PUT /identity/v3/projects/08d221c8992040aaaa2a2ecce4f55e79/users/fa24fe2ccdba4661813bcb962fd9ea5f/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:26:13.261608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4e68-4ce8-478d-b5f4-df7ee8577275 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:26:13.261983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4e68-4ce8-478d-b5f4-df7ee8577275 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:26:13.262266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4e68-4ce8-478d-b5f4-df7ee8577275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:26:13.291771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7e1b4e68-4ce8-478d-b5f4-df7ee8577275 None None] MFA Rules not processed for user `fa24fe2ccdba4661813bcb962fd9ea5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:26:13.293038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7e1b4e68-4ce8-478d-b5f4-df7ee8577275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:26:13.363518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1225/2460] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:26:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:26:23.187096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:26:23.360466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:28:08.281808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad7168-910d-4850-ae6c-5a1a8b2b8bdb admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:28:08.283491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-35ad7168-910d-4850-ae6c-5a1a8b2b8bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:08.359092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad7168-910d-4850-ae6c-5a1a8b2b8bdb admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:28:08.362707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:28:08.363307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:28:08.363583 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:28:08.363852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:28:08.365601 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:08.442228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:28:08.443663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:28:08.464873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1236/2461] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:28:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:28:09.530215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4e68-4ce8-478d-b5f4-df7ee8577275 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:28:09.546305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4e68-4ce8-478d-b5f4-df7ee8577275 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:28:09.557145 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2628d-1ff0-4fc8-b69f-70db4505502e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:28:09.557889 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2628d-1ff0-4fc8-b69f-70db4505502e service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:28:09.558165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2628d-1ff0-4fc8-b69f-70db4505502e service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:28:09.558435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2628d-1ff0-4fc8-b69f-70db4505502e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:28:09.560321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f0a2628d-1ff0-4fc8-b69f-70db4505502e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:09.628931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2628d-1ff0-4fc8-b69f-70db4505502e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:28:09.630520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2628d-1ff0-4fc8-b69f-70db4505502e service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:28:09.648402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1226/2462] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:28:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:28:10.694945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:28:10.696565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:10.774817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f7a60-8ee6-4872-bbdb-445ed7ef6eb2 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:28:10.779486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3d2cf-757d-40a1-8982-5dc972b4ff73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:28:10.780095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3d2cf-757d-40a1-8982-5dc972b4ff73 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:28:10.780370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3d2cf-757d-40a1-8982-5dc972b4ff73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:28:10.780640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3d2cf-757d-40a1-8982-5dc972b4ff73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:28:10.789734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3d2cf-757d-40a1-8982-5dc972b4ff73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:28:10.791281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3d2cf-757d-40a1-8982-5dc972b4ff73 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:28:10.796769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:28:10.802628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1237/2463] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:28:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:28:11.914971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd3558-3339-4b75-a0d4-617299ea4921 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:28:11.915501 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd3558-3339-4b75-a0d4-617299ea4921 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:28:11.915501 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd3558-3339-4b75-a0d4-617299ea4921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:28:11.945337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-addd3558-3339-4b75-a0d4-617299ea4921 None None] MFA Rules not processed for user `e62100a950db47738a7f0e3186e2344a`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:28:11.946572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-addd3558-3339-4b75-a0d4-617299ea4921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:11.953835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1227/2464] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:28:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:28:12.715343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3d2cf-757d-40a1-8982-5dc972b4ff73 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:28:12.716896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f3e3d2cf-757d-40a1-8982-5dc972b4ff73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:12.793265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3d2cf-757d-40a1-8982-5dc972b4ff73 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:28:12.800476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c427c6-fab4-4ce0-8159-9a95676d6c5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:28:12.801059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c427c6-fab4-4ce0-8159-9a95676d6c5c service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:28:12.801344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c427c6-fab4-4ce0-8159-9a95676d6c5c service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:28:12.801617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c427c6-fab4-4ce0-8159-9a95676d6c5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:28:12.805422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f4c427c6-fab4-4ce0-8159-9a95676d6c5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:12.818868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c427c6-fab4-4ce0-8159-9a95676d6c5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:28:12.820500 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c427c6-fab4-4ce0-8159-9a95676d6c5c service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:28:12.831221 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1238/2465] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:28:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:28:14.569524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd3558-3339-4b75-a0d4-617299ea4921 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:28:14.570999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-addd3558-3339-4b75-a0d4-617299ea4921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:14.583902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd3558-3339-4b75-a0d4-617299ea4921 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:28:14.587736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:28:14.588406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:28:14.588918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:28:14.589206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:28:14.591000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:14.601265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:28:14.602713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:28:14.611890 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1228/2466] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:28:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:28:16.225251 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c427c6-fab4-4ce0-8159-9a95676d6c5c service placement] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:28:16.235492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c427c6-fab4-4ce0-8159-9a95676d6c5c service placement] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:28:16.239078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86715f0f-6489-4815-8425-f4aef98ee1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:28:16.239660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86715f0f-6489-4815-8425-f4aef98ee1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:28:16.239935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86715f0f-6489-4815-8425-f4aef98ee1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:28:16.240203 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86715f0f-6489-4815-8425-f4aef98ee1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:28:16.241898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-86715f0f-6489-4815-8425-f4aef98ee1c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:16.255067 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86715f0f-6489-4815-8425-f4aef98ee1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:28:16.256493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86715f0f-6489-4815-8425-f4aef98ee1c1 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:28:16.267590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1239/2467] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:28:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:28:21.464701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:28:21.466844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:21.486630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976124e-938a-46fe-99f6-477cd607caf6 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:28:21.492078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca2a1a-b8aa-42a2-a6b0-409c0103fc4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:28:21.492677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca2a1a-b8aa-42a2-a6b0-409c0103fc4d service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:28:21.492989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca2a1a-b8aa-42a2-a6b0-409c0103fc4d service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:28:21.493264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca2a1a-b8aa-42a2-a6b0-409c0103fc4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:28:21.494991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2fca2a1a-b8aa-42a2-a6b0-409c0103fc4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:21.508282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca2a1a-b8aa-42a2-a6b0-409c0103fc4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:28:21.509946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca2a1a-b8aa-42a2-a6b0-409c0103fc4d service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:28:21.521427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1229/2468] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:28:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:28:25.590551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86715f0f-6489-4815-8425-f4aef98ee1c1 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:28:25.601938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86715f0f-6489-4815-8425-f4aef98ee1c1 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:28:25.607895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea3f73-9f8d-44ae-b720-2020502b4e49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:28:25.608503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea3f73-9f8d-44ae-b720-2020502b4e49 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:28:25.608777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea3f73-9f8d-44ae-b720-2020502b4e49 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:28:25.609078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea3f73-9f8d-44ae-b720-2020502b4e49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:28:25.610849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a6ea3f73-9f8d-44ae-b720-2020502b4e49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:28:25.623723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea3f73-9f8d-44ae-b720-2020502b4e49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:28:25.625356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea3f73-9f8d-44ae-b720-2020502b4e49 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:28:25.636610 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1240/2469] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:28:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:28:31.516702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=56 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:28:35.631511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=44 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:30:11.054957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca2a1a-b8aa-42a2-a6b0-409c0103fc4d service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:11.064870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca2a1a-b8aa-42a2-a6b0-409c0103fc4d service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:11.068440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175d1f8-91c6-42c4-87a9-1e6a504c897f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:11.069088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175d1f8-91c6-42c4-87a9-1e6a504c897f service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:11.069365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175d1f8-91c6-42c4-87a9-1e6a504c897f service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:11.069662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175d1f8-91c6-42c4-87a9-1e6a504c897f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:11.126074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175d1f8-91c6-42c4-87a9-1e6a504c897f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:11.127511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175d1f8-91c6-42c4-87a9-1e6a504c897f service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:11.136109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1230/2470] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:30:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:30:11.232945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea3f73-9f8d-44ae-b720-2020502b4e49 service nova] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:11.242340 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea3f73-9f8d-44ae-b720-2020502b4e49 service nova] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:11.246619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0aca71-88d4-4b1c-a3e3-28fbaf42cc94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:11.247232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aca71-88d4-4b1c-a3e3-28fbaf42cc94 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:11.247513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aca71-88d4-4b1c-a3e3-28fbaf42cc94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:11.247832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aca71-88d4-4b1c-a3e3-28fbaf42cc94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:11.249530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6e0aca71-88d4-4b1c-a3e3-28fbaf42cc94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:11.321868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0aca71-88d4-4b1c-a3e3-28fbaf42cc94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:11.323529 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0aca71-88d4-4b1c-a3e3-28fbaf42cc94 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:11.342729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1241/2471] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:30:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:30:11.384203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175d1f8-91c6-42c4-87a9-1e6a504c897f service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:11.393718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175d1f8-91c6-42c4-87a9-1e6a504c897f service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:11.398192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabecfc-5a6b-474f-946b-0c7e6ebf176c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:11.398812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabecfc-5a6b-474f-946b-0c7e6ebf176c service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:11.399730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabecfc-5a6b-474f-946b-0c7e6ebf176c service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:11.399730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabecfc-5a6b-474f-946b-0c7e6ebf176c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:11.401161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ffabecfc-5a6b-474f-946b-0c7e6ebf176c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:11.463386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabecfc-5a6b-474f-946b-0c7e6ebf176c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:11.464915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabecfc-5a6b-474f-946b-0c7e6ebf176c service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:11.474601 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1231/2472] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:30:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:30:11.501072 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0aca71-88d4-4b1c-a3e3-28fbaf42cc94 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:11.509441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0aca71-88d4-4b1c-a3e3-28fbaf42cc94 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:11.513846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecd7d9-87c4-457a-933c-81ef8de790a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:11.514435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecd7d9-87c4-457a-933c-81ef8de790a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:11.514717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecd7d9-87c4-457a-933c-81ef8de790a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:11.514988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecd7d9-87c4-457a-933c-81ef8de790a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:11.516686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b6ecd7d9-87c4-457a-933c-81ef8de790a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:11.588328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ecd7d9-87c4-457a-933c-81ef8de790a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:11.589749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ecd7d9-87c4-457a-933c-81ef8de790a3 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:11.598832 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1242/2473] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:30:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:30:12.690191 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabecfc-5a6b-474f-946b-0c7e6ebf176c service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:12.765215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabecfc-5a6b-474f-946b-0c7e6ebf176c service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:12.769714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:12.770340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:12.770618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:12.770887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] PATH_INFO: `/v3/users/38945899ee7e4f91af7fee8836cc2710` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:12.772003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38945899ee7e4f91af7fee8836cc2710)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:12.773974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:12.816177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:30:12.823228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38945899ee7e4f91af7fee8836cc2710'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:12.830263 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38945899ee7e4f91af7fee8836cc2710'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:12.831154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38945899ee7e4f91af7fee8836cc2710'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:12.838675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38945899ee7e4f91af7fee8836cc2710'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:12.839349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38945899ee7e4f91af7fee8836cc2710'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:12.857294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1232/2474] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:30:12 2022] DELETE /identity/v3/users/38945899ee7e4f91af7fee8836cc2710 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:30:12.861726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4335b59-d5d6-4065-b7e2-2b1d30a3ea23 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:12.862202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4335b59-d5d6-4065-b7e2-2b1d30a3ea23 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:12.862291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4335b59-d5d6-4065-b7e2-2b1d30a3ea23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:12.881777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d4335b59-d5d6-4065-b7e2-2b1d30a3ea23 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:30:12.882985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d4335b59-d5d6-4065-b7e2-2b1d30a3ea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:12.941964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1243/2475] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:30:12.952249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:12.953693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:13.009948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1b370-4ad6-472f-8ca9-4f93987e5394 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:13.013850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ca51e-88e1-4f5c-af43-4dc3cc9fbb08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:13.014455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ca51e-88e1-4f5c-af43-4dc3cc9fbb08 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:13.014737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ca51e-88e1-4f5c-af43-4dc3cc9fbb08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:13.015012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ca51e-88e1-4f5c-af43-4dc3cc9fbb08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:13.024086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ca51e-88e1-4f5c-af43-4dc3cc9fbb08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:13.025548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ca51e-88e1-4f5c-af43-4dc3cc9fbb08 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:13.035530 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1233/2476] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:30:13.307310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4335b59-d5d6-4065-b7e2-2b1d30a3ea23 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:13.308969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d4335b59-d5d6-4065-b7e2-2b1d30a3ea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:13.321058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4335b59-d5d6-4065-b7e2-2b1d30a3ea23 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:13.325775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:13.326402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:13.326687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:13.326961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] PATH_INFO: `/v3/projects/f934f93d8bc644de976b3fd821047fa2` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:13.328087 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f934f93d8bc644de976b3fd821047fa2)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:13.330109 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:13.332611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:30:13.381009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f934f93d8bc644de976b3fd821047fa2', 'request_id': 'req-7eca1810-f577-432e-b35a-c6fb10b64778'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.387408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f934f93d8bc644de976b3fd821047fa2', 'request_id': 'req-7eca1810-f577-432e-b35a-c6fb10b64778'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.394017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f934f93d8bc644de976b3fd821047fa2', 'request_id': 'req-7eca1810-f577-432e-b35a-c6fb10b64778'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.405685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] The token cache is being invalidate because project f934f93d8bc644de976b3fd821047fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:30:13.405974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.406599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.408074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1244/2477] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:30:13 2022] DELETE /identity/v3/projects/f934f93d8bc644de976b3fd821047fa2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:30:13.411901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ca51e-88e1-4f5c-af43-4dc3cc9fbb08 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:13.421688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ca51e-88e1-4f5c-af43-4dc3cc9fbb08 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:13.478164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:13.478828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:13.479112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:13.479390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] PATH_INFO: `/v3/users/fa24fe2ccdba4661813bcb962fd9ea5f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:13.480560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa24fe2ccdba4661813bcb962fd9ea5f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:13.482623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:13.528277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa24fe2ccdba4661813bcb962fd9ea5f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.535386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa24fe2ccdba4661813bcb962fd9ea5f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.535988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa24fe2ccdba4661813bcb962fd9ea5f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.541338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:30:13.543511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa24fe2ccdba4661813bcb962fd9ea5f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.544442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-28cac082-d1b1-48b5-9a08-80ed9774e351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa24fe2ccdba4661813bcb962fd9ea5f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.564442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1234/2478] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:30:13 2022] DELETE /identity/v3/users/fa24fe2ccdba4661813bcb962fd9ea5f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:30:13.834603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:13.836039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:13.850075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca1810-f577-432e-b35a-c6fb10b64778 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:13.908769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:13.909470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:13.909704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:13.909974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] PATH_INFO: `/v3/projects/08d221c8992040aaaa2a2ecce4f55e79` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:13.911132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d221c8992040aaaa2a2ecce4f55e79)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:13.913095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:13.959197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d221c8992040aaaa2a2ecce4f55e79', 'request_id': 'req-eb023da1-a980-416c-9ed0-860db942d0e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.965384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d221c8992040aaaa2a2ecce4f55e79', 'request_id': 'req-eb023da1-a980-416c-9ed0-860db942d0e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.972842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d221c8992040aaaa2a2ecce4f55e79', 'request_id': 'req-eb023da1-a980-416c-9ed0-860db942d0e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.985302 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] The token cache is being invalidate because project 08d221c8992040aaaa2a2ecce4f55e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:30:13.985624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.986264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-eb023da1-a980-416c-9ed0-860db942d0e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:30:13.987752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1245/2479] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:30:13 2022] DELETE /identity/v3/projects/08d221c8992040aaaa2a2ecce4f55e79 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:30:13.994192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abff76d-9ac2-4499-88c8-dcd20a57dc20 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:13.994682 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abff76d-9ac2-4499-88c8-dcd20a57dc20 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:13.994782 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abff76d-9ac2-4499-88c8-dcd20a57dc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.014429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8abff76d-9ac2-4499-88c8-dcd20a57dc20 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:30:14.015607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8abff76d-9ac2-4499-88c8-dcd20a57dc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:14.073933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1235/2480] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:30:14.082026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd31a01-705b-49c3-a0bb-0baf6398496a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.082348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd31a01-705b-49c3-a0bb-0baf6398496a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.082628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd31a01-705b-49c3-a0bb-0baf6398496a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.103366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-0bd31a01-705b-49c3-a0bb-0baf6398496a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:30:14.104585 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0bd31a01-705b-49c3-a0bb-0baf6398496a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:14.113164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1246/2481] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:30:14.117314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abff76d-9ac2-4499-88c8-dcd20a57dc20 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.127653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abff76d-9ac2-4499-88c8-dcd20a57dc20 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.133973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2f408-53e8-4760-bc6e-bf2d8316f5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.134597 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2f408-53e8-4760-bc6e-bf2d8316f5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.134881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2f408-53e8-4760-bc6e-bf2d8316f5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.135158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2f408-53e8-4760-bc6e-bf2d8316f5b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.135960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2f408-53e8-4760-bc6e-bf2d8316f5b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:30:14.136271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2f408-53e8-4760-bc6e-bf2d8316f5b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.138615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2f408-53e8-4760-bc6e-bf2d8316f5b8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.150637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1236/2482] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 11:30:14.156360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd31a01-705b-49c3-a0bb-0baf6398496a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.165661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd31a01-705b-49c3-a0bb-0baf6398496a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.171322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69a3d7-8b68-471b-8795-526f200ca43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.171910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69a3d7-8b68-471b-8795-526f200ca43d admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.172184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69a3d7-8b68-471b-8795-526f200ca43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.172447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69a3d7-8b68-471b-8795-526f200ca43d admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.173106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69a3d7-8b68-471b-8795-526f200ca43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.175010 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69a3d7-8b68-471b-8795-526f200ca43d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.191948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:30:14.205909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1247/2483] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:30:14 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:30:14.210033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2f408-53e8-4760-bc6e-bf2d8316f5b8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.221016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2f408-53e8-4760-bc6e-bf2d8316f5b8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.275735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e3884-cd6d-4f27-9c85-cc174d283320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.276343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e3884-cd6d-4f27-9c85-cc174d283320 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.276623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e3884-cd6d-4f27-9c85-cc174d283320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.276929 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e3884-cd6d-4f27-9c85-cc174d283320 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.278165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e3884-cd6d-4f27-9c85-cc174d283320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.280103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e3884-cd6d-4f27-9c85-cc174d283320 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.300728 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1237/2484] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:30:14 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:30:14.304510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69a3d7-8b68-471b-8795-526f200ca43d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.315076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69a3d7-8b68-471b-8795-526f200ca43d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.319623 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dbf5e-6f1f-4385-b41a-21b28e2f553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.320212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dbf5e-6f1f-4385-b41a-21b28e2f553e admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.320486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dbf5e-6f1f-4385-b41a-21b28e2f553e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.320753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dbf5e-6f1f-4385-b41a-21b28e2f553e admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.321414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dbf5e-6f1f-4385-b41a-21b28e2f553e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.322994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dbf5e-6f1f-4385-b41a-21b28e2f553e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.333422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1248/2485] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:30:14.337180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e3884-cd6d-4f27-9c85-cc174d283320 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.347834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e3884-cd6d-4f27-9c85-cc174d283320 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.352462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8053d-2a45-4b72-b348-7972c48c2f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.353228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8053d-2a45-4b72-b348-7972c48c2f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.353744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8053d-2a45-4b72-b348-7972c48c2f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.354211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8053d-2a45-4b72-b348-7972c48c2f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.354942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8053d-2a45-4b72-b348-7972c48c2f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.356600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8053d-2a45-4b72-b348-7972c48c2f55 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.367054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1238/2486] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:30:14.371934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dbf5e-6f1f-4385-b41a-21b28e2f553e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.388125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dbf5e-6f1f-4385-b41a-21b28e2f553e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.393894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c88f0-19b3-4c47-9b92-b8a55657323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.394550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c88f0-19b3-4c47-9b92-b8a55657323f admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.394834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c88f0-19b3-4c47-9b92-b8a55657323f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.395153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c88f0-19b3-4c47-9b92-b8a55657323f admin admin] PATH_INFO: `/v3/projects/88f81596daa34cb8b713727dad9273da/users/81609ad274ce43ee8c62da731d2f17f0/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.411130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c88f0-19b3-4c47-9b92-b8a55657323f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f81596daa34cb8b713727dad9273da, user_id=81609ad274ce43ee8c62da731d2f17f0, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.413845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c88f0-19b3-4c47-9b92-b8a55657323f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.426068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1249/2487] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:30:14 2022] PUT /identity/v3/projects/88f81596daa34cb8b713727dad9273da/users/81609ad274ce43ee8c62da731d2f17f0/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:30:14.436928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0604d24-0c1c-44ab-8f94-c1d963ca0499 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.437223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0604d24-0c1c-44ab-8f94-c1d963ca0499 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.437503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0604d24-0c1c-44ab-8f94-c1d963ca0499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.466440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d0604d24-0c1c-44ab-8f94-c1d963ca0499 None None] MFA Rules not processed for user `81609ad274ce43ee8c62da731d2f17f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:30:14.467671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d0604d24-0c1c-44ab-8f94-c1d963ca0499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:14.501264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:30:14.535018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1239/2488] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:30:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:30:14.539664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c88f0-19b3-4c47-9b92-b8a55657323f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.602339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c88f0-19b3-4c47-9b92-b8a55657323f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.607247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb66594-45a1-4ed7-928e-02e7c01fc9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.607845 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb66594-45a1-4ed7-928e-02e7c01fc9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.608125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb66594-45a1-4ed7-928e-02e7c01fc9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.608397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb66594-45a1-4ed7-928e-02e7c01fc9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.609092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb66594-45a1-4ed7-928e-02e7c01fc9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.611005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb66594-45a1-4ed7-928e-02e7c01fc9b2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.638144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1250/2489] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:30:14 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:30:14.642223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0604d24-0c1c-44ab-8f94-c1d963ca0499 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.653167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0604d24-0c1c-44ab-8f94-c1d963ca0499 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.707982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed35a0-3464-467f-b604-9ff56dc75162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.708578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed35a0-3464-467f-b604-9ff56dc75162 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.708882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed35a0-3464-467f-b604-9ff56dc75162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.709153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed35a0-3464-467f-b604-9ff56dc75162 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.709776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed35a0-3464-467f-b604-9ff56dc75162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.711701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed35a0-3464-467f-b604-9ff56dc75162 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.732077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1240/2490] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:30:14 2022] POST /identity/v3/users => generated 301 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:30:14.735969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb66594-45a1-4ed7-928e-02e7c01fc9b2 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.746684 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb66594-45a1-4ed7-928e-02e7c01fc9b2 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.751440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70b0e-f308-4c26-972e-a3c23307e386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.752048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70b0e-f308-4c26-972e-a3c23307e386 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.752332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70b0e-f308-4c26-972e-a3c23307e386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.752606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70b0e-f308-4c26-972e-a3c23307e386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.753289 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70b0e-f308-4c26-972e-a3c23307e386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.754902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70b0e-f308-4c26-972e-a3c23307e386 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.767483 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1251/2491] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:30:14.771640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed35a0-3464-467f-b604-9ff56dc75162 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.781993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed35a0-3464-467f-b604-9ff56dc75162 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.786939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b58d6-0e62-4d08-97b2-8e3c7c191ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.787655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b58d6-0e62-4d08-97b2-8e3c7c191ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.788015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b58d6-0e62-4d08-97b2-8e3c7c191ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.788315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b58d6-0e62-4d08-97b2-8e3c7c191ae0 admin admin] PATH_INFO: `/v3/projects/fcbff1c89d99433191a23cc5b22bfba5/users/4aacea0bc3244509bbeb3ae2e9e7e26f/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.803206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b58d6-0e62-4d08-97b2-8e3c7c191ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbff1c89d99433191a23cc5b22bfba5, user_id=4aacea0bc3244509bbeb3ae2e9e7e26f, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.805917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b58d6-0e62-4d08-97b2-8e3c7c191ae0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.819545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1241/2492] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:30:14 2022] PUT /identity/v3/projects/fcbff1c89d99433191a23cc5b22bfba5/users/4aacea0bc3244509bbeb3ae2e9e7e26f/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:30:14.823505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70b0e-f308-4c26-972e-a3c23307e386 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.834290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70b0e-f308-4c26-972e-a3c23307e386 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.890792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc79451-c323-470c-8b7b-dd6d31ca9d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.891394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc79451-c323-470c-8b7b-dd6d31ca9d3b admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.891685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc79451-c323-470c-8b7b-dd6d31ca9d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.891947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc79451-c323-470c-8b7b-dd6d31ca9d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.892572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc79451-c323-470c-8b7b-dd6d31ca9d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.894214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc79451-c323-470c-8b7b-dd6d31ca9d3b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.903860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1252/2493] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:30:14.907988 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b58d6-0e62-4d08-97b2-8e3c7c191ae0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:14.918657 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b58d6-0e62-4d08-97b2-8e3c7c191ae0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:14.923412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a739b3-25c7-4de7-8663-13a226d3d79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:14.924084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a739b3-25c7-4de7-8663-13a226d3d79a admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.924366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a739b3-25c7-4de7-8663-13a226d3d79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.924637 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a739b3-25c7-4de7-8663-13a226d3d79a admin admin] PATH_INFO: `/v3/projects/fcbff1c89d99433191a23cc5b22bfba5/users/4aacea0bc3244509bbeb3ae2e9e7e26f/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.926672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a739b3-25c7-4de7-8663-13a226d3d79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbff1c89d99433191a23cc5b22bfba5, user_id=4aacea0bc3244509bbeb3ae2e9e7e26f, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:14.929744 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a739b3-25c7-4de7-8663-13a226d3d79a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:14.941795 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1242/2494] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:30:14 2022] PUT /identity/v3/projects/fcbff1c89d99433191a23cc5b22bfba5/users/4aacea0bc3244509bbeb3ae2e9e7e26f/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:30:14.954593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e5457-9035-4901-a6b8-d494a23e231a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:14.955133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e5457-9035-4901-a6b8-d494a23e231a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:14.955133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e5457-9035-4901-a6b8-d494a23e231a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:14.983426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2f7e5457-9035-4901-a6b8-d494a23e231a None None] MFA Rules not processed for user `4aacea0bc3244509bbeb3ae2e9e7e26f`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:30:14.984629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2f7e5457-9035-4901-a6b8-d494a23e231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:15.001784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:30:15.053568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1253/2495] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:30:14 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:30:24.937697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=56,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:30:25.050663 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=21 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:30:48.964086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a739b3-25c7-4de7-8663-13a226d3d79a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:30:48.965897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-40a739b3-25c7-4de7-8663-13a226d3d79a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:49.041018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a739b3-25c7-4de7-8663-13a226d3d79a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:30:49.045940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c6326-64b3-451e-bb6a-a86f00630d01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:30:49.046536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c6326-64b3-451e-bb6a-a86f00630d01 service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:30:49.046814 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c6326-64b3-451e-bb6a-a86f00630d01 service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:30:49.047085 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c6326-64b3-451e-bb6a-a86f00630d01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:30:49.048773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-608c6326-64b3-451e-bb6a-a86f00630d01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:30:49.061048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c6326-64b3-451e-bb6a-a86f00630d01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:30:49.062713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c6326-64b3-451e-bb6a-a86f00630d01 service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:30:49.073549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1243/2496] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:30:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:30:59.068455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=32 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:08.079571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e5457-9035-4901-a6b8-d494a23e231a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:08.080983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2f7e5457-9035-4901-a6b8-d494a23e231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:08.138242 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e5457-9035-4901-a6b8-d494a23e231a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:08.141787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd288ca-f156-4db4-9276-4d981782de80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:08.142391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd288ca-f156-4db4-9276-4d981782de80 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:08.142669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd288ca-f156-4db4-9276-4d981782de80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:08.142940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd288ca-f156-4db4-9276-4d981782de80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:08.152422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd288ca-f156-4db4-9276-4d981782de80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:08.153893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd288ca-f156-4db4-9276-4d981782de80 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:08.163205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1254/2497] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:08.492351 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c6326-64b3-451e-bb6a-a86f00630d01 service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:08.568159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c6326-64b3-451e-bb6a-a86f00630d01 service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:08.573262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:08.573931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:08.574214 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:08.574489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] PATH_INFO: `/v3/users/81609ad274ce43ee8c62da731d2f17f0` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:08.575619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81609ad274ce43ee8c62da731d2f17f0)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:08.577625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:08.623672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81609ad274ce43ee8c62da731d2f17f0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:08.631483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81609ad274ce43ee8c62da731d2f17f0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:08.632120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81609ad274ce43ee8c62da731d2f17f0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:08.639986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81609ad274ce43ee8c62da731d2f17f0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:08.640670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81609ad274ce43ee8c62da731d2f17f0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:08.660702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1244/2498] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:08 2022] DELETE /identity/v3/users/81609ad274ce43ee8c62da731d2f17f0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:08.665220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2210e8-5d3c-4745-a424-b943a9cf79ed None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:08.665503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2210e8-5d3c-4745-a424-b943a9cf79ed None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:08.666474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2210e8-5d3c-4745-a424-b943a9cf79ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:08.687581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-ef2210e8-5d3c-4745-a424-b943a9cf79ed None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:08.689243 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ef2210e8-5d3c-4745-a424-b943a9cf79ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:08.756145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1255/2499] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:08.765844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:08.767255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:08.843132 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11707c93-39e7-4dab-ade7-729940c3cab5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:08.847309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fc89f-fd7f-4b6a-9247-69847899acbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:08.847930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fc89f-fd7f-4b6a-9247-69847899acbe service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:08.848211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fc89f-fd7f-4b6a-9247-69847899acbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:08.848487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fc89f-fd7f-4b6a-9247-69847899acbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:08.858658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fc89f-fd7f-4b6a-9247-69847899acbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:08.860157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fc89f-fd7f-4b6a-9247-69847899acbe service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:08.871006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1245/2500] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:08.974452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2210e8-5d3c-4745-a424-b943a9cf79ed None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:08.976044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ef2210e8-5d3c-4745-a424-b943a9cf79ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:09.052334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fc89f-fd7f-4b6a-9247-69847899acbe service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:09.055336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c07fc89f-fd7f-4b6a-9247-69847899acbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:09.089386 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2210e8-5d3c-4745-a424-b943a9cf79ed None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:09.095593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519a0f0-27d4-42ab-8a62-5754416d1df5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:09.096231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519a0f0-27d4-42ab-8a62-5754416d1df5 service glance] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:09.096518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519a0f0-27d4-42ab-8a62-5754416d1df5 service glance] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:09.096818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519a0f0-27d4-42ab-8a62-5754416d1df5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:09.098834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4519a0f0-27d4-42ab-8a62-5754416d1df5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:09.100285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-4519a0f0-27d4-42ab-8a62-5754416d1df5 service glance] Could not find project: c63dfe497afb46ce864be33c80d38af5.: keystone.exception.ProjectNotFound: Could not find project: c63dfe497afb46ce864be33c80d38af5. Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 15 11:32:09.115685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 11:32:09.117055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 11:32:09.119260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 11:32:09.120750 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 11:32:09.122073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c63dfe497afb46ce864be33c80d38af5. Sep 15 11:32:09.122073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 11:32:09.122073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1256/2501] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:08 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 15 11:32:09.173569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519a0f0-27d4-42ab-8a62-5754416d1df5 service glance] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:09.195030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519a0f0-27d4-42ab-8a62-5754416d1df5 service glance] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:09.195809 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fc89f-fd7f-4b6a-9247-69847899acbe service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:09.202592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421aa4b-3244-4369-968b-b6da5fa99cfd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:09.203369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421aa4b-3244-4369-968b-b6da5fa99cfd service glance] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:09.203705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421aa4b-3244-4369-968b-b6da5fa99cfd service glance] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:09.204299 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421aa4b-3244-4369-968b-b6da5fa99cfd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:09.206763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4421aa4b-3244-4369-968b-b6da5fa99cfd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:09.212095 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aa740-33fb-4689-80d9-3625fb74e8e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:09.212847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aa740-33fb-4689-80d9-3625fb74e8e1 service glance] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:09.213943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aa740-33fb-4689-80d9-3625fb74e8e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:09.215265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aa740-33fb-4689-80d9-3625fb74e8e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:09.218424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-562aa740-33fb-4689-80d9-3625fb74e8e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application [None req-562aa740-33fb-4689-80d9-3625fb74e8e1 service glance] Could not find project: c92e1f7a020b4269bfd9c3e9fdee95a6.: keystone.exception.ProjectNotFound: Could not find project: c92e1f7a020b4269bfd9c3e9fdee95a6. Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 15 11:32:09.233724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 11:32:09.234786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 11:32:09.235900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application with Lock( Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._enter() Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.creator() Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application created_value = creator( Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 11:32:09.237474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 11:32:09.238970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c92e1f7a020b4269bfd9c3e9fdee95a6. Sep 15 11:32:09.238970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: ERROR keystone.server.flask.application Sep 15 11:32:09.241244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1257/2502] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:09 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 15 11:32:09.243344 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aa740-33fb-4689-80d9-3625fb74e8e1 service glance] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:09.245555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-562aa740-33fb-4689-80d9-3625fb74e8e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-4421aa4b-3244-4369-968b-b6da5fa99cfd service glance] Could not find project: 1c32818de9364d95af65290e4dd9f658.: keystone.exception.ProjectNotFound: Could not find project: 1c32818de9364d95af65290e4dd9f658. Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 15 11:32:09.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 11:32:09.252935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 11:32:09.253932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 11:32:09.255075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 11:32:09.257572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 1c32818de9364d95af65290e4dd9f658. Sep 15 11:32:09.257572 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 15 11:32:09.262422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aa740-33fb-4689-80d9-3625fb74e8e1 service glance] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:09.262891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1246/2503] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:09 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 15 11:32:09.269395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:09.270403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:09.271157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:09.271442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] PATH_INFO: `/v3/projects/88f81596daa34cb8b713727dad9273da` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:09.274618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f81596daa34cb8b713727dad9273da)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:09.277156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:09.300214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:09.378556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f81596daa34cb8b713727dad9273da', 'request_id': 'req-8d301fdc-379d-42a1-8b84-19491bf1b565'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.389740 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f81596daa34cb8b713727dad9273da', 'request_id': 'req-8d301fdc-379d-42a1-8b84-19491bf1b565'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.403098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f81596daa34cb8b713727dad9273da', 'request_id': 'req-8d301fdc-379d-42a1-8b84-19491bf1b565'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.437298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] The token cache is being invalidate because project 88f81596daa34cb8b713727dad9273da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:09.437826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.438577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.440207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1258/2504] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:09 2022] DELETE /identity/v3/projects/88f81596daa34cb8b713727dad9273da => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:09.444218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421aa4b-3244-4369-968b-b6da5fa99cfd service glance] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:09.458238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421aa4b-3244-4369-968b-b6da5fa99cfd service glance] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:09.485576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:09.487918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:09.564352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:09.565051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:09.565336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:09.565608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] PATH_INFO: `/v3/users/4aacea0bc3244509bbeb3ae2e9e7e26f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:09.566928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aacea0bc3244509bbeb3ae2e9e7e26f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:09.567278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d301fdc-379d-42a1-8b84-19491bf1b565 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:09.569399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:09.574382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:09.575375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:09.576487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:09.576865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:09.592167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:09.611651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:09.615160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:09.628264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1259/2505] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:09.639894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aacea0bc3244509bbeb3ae2e9e7e26f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.648652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aacea0bc3244509bbeb3ae2e9e7e26f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.649445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aacea0bc3244509bbeb3ae2e9e7e26f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.659720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aacea0bc3244509bbeb3ae2e9e7e26f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.660651 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aacea0bc3244509bbeb3ae2e9e7e26f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:09.685881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1247/2506] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:09 2022] DELETE /identity/v3/users/4aacea0bc3244509bbeb3ae2e9e7e26f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:09.838223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:09.838967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:09.948575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf41162-4330-4a31-827e-c90cd901097d service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:10.019332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:10.047707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad29a8e-fc6d-4ba1-b6d1-636351102a8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:10.048672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad29a8e-fc6d-4ba1-b6d1-636351102a8d service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:10.048882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad29a8e-fc6d-4ba1-b6d1-636351102a8d service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:10.049263 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad29a8e-fc6d-4ba1-b6d1-636351102a8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:10.051082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cad29a8e-fc6d-4ba1-b6d1-636351102a8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:10.064436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad29a8e-fc6d-4ba1-b6d1-636351102a8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:10.066561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad29a8e-fc6d-4ba1-b6d1-636351102a8d service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:10.080059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1260/2507] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:10.333964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:10.338996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:10.409575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:10.448686 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f26bb2-6c27-4e62-835e-963afd288d8f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:10.456257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:10.458417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:10.458600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:10.458993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] PATH_INFO: `/v3/projects/fcbff1c89d99433191a23cc5b22bfba5` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:10.461022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcbff1c89d99433191a23cc5b22bfba5)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:10.465059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:10.577083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcbff1c89d99433191a23cc5b22bfba5', 'request_id': 'req-feeea742-cad3-49d1-b0dc-1337b99ceee8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:10.589700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcbff1c89d99433191a23cc5b22bfba5', 'request_id': 'req-feeea742-cad3-49d1-b0dc-1337b99ceee8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:10.602872 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcbff1c89d99433191a23cc5b22bfba5', 'request_id': 'req-feeea742-cad3-49d1-b0dc-1337b99ceee8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:10.623309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] The token cache is being invalidate because project fcbff1c89d99433191a23cc5b22bfba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:10.623613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:10.624285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:10.626175 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1248/2508] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:10 2022] DELETE /identity/v3/projects/fcbff1c89d99433191a23cc5b22bfba5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:10.630791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971b7c8-4bbe-428a-a681-17282ca2e618 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:10.632512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971b7c8-4bbe-428a-a681-17282ca2e618 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:10.635222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971b7c8-4bbe-428a-a681-17282ca2e618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:10.654900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:10.657769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:10.680253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-7971b7c8-4bbe-428a-a681-17282ca2e618 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:10.682768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7971b7c8-4bbe-428a-a681-17282ca2e618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:10.779408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeea742-cad3-49d1-b0dc-1337b99ceee8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:10.784877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5659f2-4720-4f83-b2a6-0290de5830f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:10.786438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5659f2-4720-4f83-b2a6-0290de5830f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:10.786818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5659f2-4720-4f83-b2a6-0290de5830f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:10.787095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5659f2-4720-4f83-b2a6-0290de5830f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:10.790185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bc5659f2-4720-4f83-b2a6-0290de5830f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:10.802705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1261/2509] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:10.808327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971b7c8-4bbe-428a-a681-17282ca2e618 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:10.823958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971b7c8-4bbe-428a-a681-17282ca2e618 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:10.831071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfad4de-b8bb-4859-89f8-eafeb390242b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:10.831862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfad4de-b8bb-4859-89f8-eafeb390242b service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:10.832248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfad4de-b8bb-4859-89f8-eafeb390242b service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:10.832630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfad4de-b8bb-4859-89f8-eafeb390242b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:10.834907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6bfad4de-b8bb-4859-89f8-eafeb390242b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:10.849641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfad4de-b8bb-4859-89f8-eafeb390242b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:10.851081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfad4de-b8bb-4859-89f8-eafeb390242b service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:10.861298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1262/2510] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:10.862551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d15e80-7390-4571-867d-f184afdcd114 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:10.862847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d15e80-7390-4571-867d-f184afdcd114 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:10.863121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d15e80-7390-4571-867d-f184afdcd114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:10.869355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5659f2-4720-4f83-b2a6-0290de5830f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:10.870974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5659f2-4720-4f83-b2a6-0290de5830f2 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:10.882397 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1249/2511] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:10.883912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-21d15e80-7390-4571-867d-f184afdcd114 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:10.885396 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-21d15e80-7390-4571-867d-f184afdcd114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:10.894856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1263/2512] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:10.899038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5659f2-4720-4f83-b2a6-0290de5830f2 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:10.910116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5659f2-4720-4f83-b2a6-0290de5830f2 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:10.915059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8405e4-6602-46be-b0e8-3d4bb52c208c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:10.916323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8405e4-6602-46be-b0e8-3d4bb52c208c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:10.916820 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8405e4-6602-46be-b0e8-3d4bb52c208c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:10.917216 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8405e4-6602-46be-b0e8-3d4bb52c208c admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:10.917874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8405e4-6602-46be-b0e8-3d4bb52c208c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:10.918182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8405e4-6602-46be-b0e8-3d4bb52c208c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:10.919812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8405e4-6602-46be-b0e8-3d4bb52c208c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:10.932583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1250/2513] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:10.937003 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d15e80-7390-4571-867d-f184afdcd114 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:10.946710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d15e80-7390-4571-867d-f184afdcd114 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:10.951732 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82c60d-86cf-4dad-bb7d-0ad9671b76b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:10.952333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82c60d-86cf-4dad-bb7d-0ad9671b76b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:10.952608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82c60d-86cf-4dad-bb7d-0ad9671b76b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:10.952909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82c60d-86cf-4dad-bb7d-0ad9671b76b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:10.953552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82c60d-86cf-4dad-bb7d-0ad9671b76b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:10.955509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82c60d-86cf-4dad-bb7d-0ad9671b76b7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:10.987657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1264/2514] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:10 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:10.991778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8405e4-6602-46be-b0e8-3d4bb52c208c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.000266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:11.004437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8405e4-6602-46be-b0e8-3d4bb52c208c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.072969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63af39-f9e8-41f3-af34-4302f2335c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.073634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63af39-f9e8-41f3-af34-4302f2335c0c admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.073980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63af39-f9e8-41f3-af34-4302f2335c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.074321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63af39-f9e8-41f3-af34-4302f2335c0c admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.075036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63af39-f9e8-41f3-af34-4302f2335c0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.077083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63af39-f9e8-41f3-af34-4302f2335c0c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.103655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1251/2515] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:10 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:11.105139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82c60d-86cf-4dad-bb7d-0ad9671b76b7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.115259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82c60d-86cf-4dad-bb7d-0ad9671b76b7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.120043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3efb85-3a08-47b3-a87d-b8f000176c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.120643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3efb85-3a08-47b3-a87d-b8f000176c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.120957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3efb85-3a08-47b3-a87d-b8f000176c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.121227 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3efb85-3a08-47b3-a87d-b8f000176c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.121860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3efb85-3a08-47b3-a87d-b8f000176c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.123467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3efb85-3a08-47b3-a87d-b8f000176c5c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.133562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1265/2516] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:11.138200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63af39-f9e8-41f3-af34-4302f2335c0c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.148830 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63af39-f9e8-41f3-af34-4302f2335c0c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.154646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9a918-6d1f-4b34-b861-cb76ea726ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.155270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9a918-6d1f-4b34-b861-cb76ea726ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.155555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9a918-6d1f-4b34-b861-cb76ea726ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.155828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9a918-6d1f-4b34-b861-cb76ea726ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.156476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9a918-6d1f-4b34-b861-cb76ea726ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.158220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9a918-6d1f-4b34-b861-cb76ea726ed7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.169441 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1252/2517] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:11.173643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3efb85-3a08-47b3-a87d-b8f000176c5c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.184060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3efb85-3a08-47b3-a87d-b8f000176c5c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.189133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d0aa3-89b9-4668-bded-9984b8d1c852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.189790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d0aa3-89b9-4668-bded-9984b8d1c852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.190070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d0aa3-89b9-4668-bded-9984b8d1c852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.190343 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d0aa3-89b9-4668-bded-9984b8d1c852 admin admin] PATH_INFO: `/v3/projects/d103c5ff00a6461d914e628fa6f094be/users/6889363e2cae47ada752e74aef71f126/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.205919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d0aa3-89b9-4668-bded-9984b8d1c852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d103c5ff00a6461d914e628fa6f094be, user_id=6889363e2cae47ada752e74aef71f126, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.208591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d0aa3-89b9-4668-bded-9984b8d1c852 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.223920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1266/2518] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:11 2022] PUT /identity/v3/projects/d103c5ff00a6461d914e628fa6f094be/users/6889363e2cae47ada752e74aef71f126/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:11.236962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474050-2228-44de-8cea-19fca87e5e0f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.237255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474050-2228-44de-8cea-19fca87e5e0f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.237532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474050-2228-44de-8cea-19fca87e5e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.266378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-11474050-2228-44de-8cea-19fca87e5e0f None None] MFA Rules not processed for user `6889363e2cae47ada752e74aef71f126`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:11.267631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-11474050-2228-44de-8cea-19fca87e5e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:11.339566 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1253/2519] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:11.345025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d0aa3-89b9-4668-bded-9984b8d1c852 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.406722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:11.430523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d0aa3-89b9-4668-bded-9984b8d1c852 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.436783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe528788-26a0-4107-a306-c70fceef84b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.438012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe528788-26a0-4107-a306-c70fceef84b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.438362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe528788-26a0-4107-a306-c70fceef84b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.438676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe528788-26a0-4107-a306-c70fceef84b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.439355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe528788-26a0-4107-a306-c70fceef84b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.441510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe528788-26a0-4107-a306-c70fceef84b7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.475969 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1267/2520] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:11 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:11.476923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474050-2228-44de-8cea-19fca87e5e0f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.490945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474050-2228-44de-8cea-19fca87e5e0f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.558914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71952-6162-432f-bfc7-94a242f895e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.559816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71952-6162-432f-bfc7-94a242f895e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.560375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71952-6162-432f-bfc7-94a242f895e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.560893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71952-6162-432f-bfc7-94a242f895e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.561702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa71952-6162-432f-bfc7-94a242f895e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.564341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa71952-6162-432f-bfc7-94a242f895e9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.591147 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1254/2521] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:11 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:11.595081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe528788-26a0-4107-a306-c70fceef84b7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.607592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe528788-26a0-4107-a306-c70fceef84b7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.613236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b18e70-53f6-4f8f-9233-e56f467d9a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.614299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18e70-53f6-4f8f-9233-e56f467d9a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.614422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18e70-53f6-4f8f-9233-e56f467d9a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.614690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18e70-53f6-4f8f-9233-e56f467d9a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.615337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b18e70-53f6-4f8f-9233-e56f467d9a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.617748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b18e70-53f6-4f8f-9233-e56f467d9a81 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.635574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1268/2522] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:11.637011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71952-6162-432f-bfc7-94a242f895e9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.644372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:11.653578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71952-6162-432f-bfc7-94a242f895e9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.661869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4eeb-b06b-4a0c-be9d-85d8a2d32c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.662646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4eeb-b06b-4a0c-be9d-85d8a2d32c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.662981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4eeb-b06b-4a0c-be9d-85d8a2d32c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.663274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4eeb-b06b-4a0c-be9d-85d8a2d32c6a admin admin] PATH_INFO: `/v3/projects/aba9af6e9e1e420ba17ce00bb85e68bc/users/fd6012ffaf5b4a0da4926eba0f7d5ab7/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.694011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed4eeb-b06b-4a0c-be9d-85d8a2d32c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba9af6e9e1e420ba17ce00bb85e68bc, user_id=fd6012ffaf5b4a0da4926eba0f7d5ab7, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.696941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed4eeb-b06b-4a0c-be9d-85d8a2d32c6a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.717195 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1255/2523] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:11 2022] PUT /identity/v3/projects/aba9af6e9e1e420ba17ce00bb85e68bc/users/fd6012ffaf5b4a0da4926eba0f7d5ab7/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:11.718404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b18e70-53f6-4f8f-9233-e56f467d9a81 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.730493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b18e70-53f6-4f8f-9233-e56f467d9a81 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.798641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18f535-a14d-4964-bc48-0f31dcd08ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.799573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18f535-a14d-4964-bc48-0f31dcd08ace admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.800011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18f535-a14d-4964-bc48-0f31dcd08ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.800430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18f535-a14d-4964-bc48-0f31dcd08ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.801285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18f535-a14d-4964-bc48-0f31dcd08ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.804256 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18f535-a14d-4964-bc48-0f31dcd08ace admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.817291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1269/2524] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:11.824333 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4eeb-b06b-4a0c-be9d-85d8a2d32c6a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.839399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4eeb-b06b-4a0c-be9d-85d8a2d32c6a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.848887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18f535-a14d-4964-bc48-0f31dcd08ace admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:11.849110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb8b08-b5fb-44a7-a6a0-e59b64deafbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:11.850334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb8b08-b5fb-44a7-a6a0-e59b64deafbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.850629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb8b08-b5fb-44a7-a6a0-e59b64deafbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.850849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6c18f535-a14d-4964-bc48-0f31dcd08ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:11.850961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb8b08-b5fb-44a7-a6a0-e59b64deafbe admin admin] PATH_INFO: `/v3/projects/aba9af6e9e1e420ba17ce00bb85e68bc/users/fd6012ffaf5b4a0da4926eba0f7d5ab7/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.855217 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb8b08-b5fb-44a7-a6a0-e59b64deafbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba9af6e9e1e420ba17ce00bb85e68bc, user_id=fd6012ffaf5b4a0da4926eba0f7d5ab7, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:11.859972 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb8b08-b5fb-44a7-a6a0-e59b64deafbe admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:11.887763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1256/2525] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:11 2022] PUT /identity/v3/projects/aba9af6e9e1e420ba17ce00bb85e68bc/users/fd6012ffaf5b4a0da4926eba0f7d5ab7/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:11.889646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94701a-64fc-4c53-b51e-10c9dd9ee7ab None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:11.890223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94701a-64fc-4c53-b51e-10c9dd9ee7ab None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:11.890629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94701a-64fc-4c53-b51e-10c9dd9ee7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:11.949566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18f535-a14d-4964-bc48-0f31dcd08ace admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:11.952227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0a94701a-64fc-4c53-b51e-10c9dd9ee7ab None None] MFA Rules not processed for user `fd6012ffaf5b4a0da4926eba0f7d5ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:11.953472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0a94701a-64fc-4c53-b51e-10c9dd9ee7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:12.014685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:12.026011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:12.026645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:12.026927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:12.027204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:12.029061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:12.043111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:12.047608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1257/2526] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:12.047608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94701a-64fc-4c53-b51e-10c9dd9ee7ab None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:12.047803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:12.047879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0a94701a-64fc-4c53-b51e-10c9dd9ee7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:12.057038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1270/2527] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:12.059708 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:12.061415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:12.130786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad639c0-71d2-4830-88d2-06db21044306 service placement] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:12.135490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371f29d-15fe-4f07-81f5-e71eb00f9186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:12.136269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371f29d-15fe-4f07-81f5-e71eb00f9186 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:12.136551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371f29d-15fe-4f07-81f5-e71eb00f9186 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:12.136847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371f29d-15fe-4f07-81f5-e71eb00f9186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:12.145612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94701a-64fc-4c53-b51e-10c9dd9ee7ab None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:12.146875 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371f29d-15fe-4f07-81f5-e71eb00f9186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:12.148354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371f29d-15fe-4f07-81f5-e71eb00f9186 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:12.151135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:12.151965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:12.152247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:12.152518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] PATH_INFO: `/v3/users/f4ecdf8ab7d84d35adb866f94aaf3eff` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:12.154644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ecdf8ab7d84d35adb866f94aaf3eff)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:12.157057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:12.159377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1271/2528] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:12.213091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ecdf8ab7d84d35adb866f94aaf3eff'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.221777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:12.226471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ecdf8ab7d84d35adb866f94aaf3eff'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.228548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ecdf8ab7d84d35adb866f94aaf3eff'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.238945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ecdf8ab7d84d35adb866f94aaf3eff'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.239705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ecdf8ab7d84d35adb866f94aaf3eff'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.260848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1258/2529] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:12 2022] DELETE /identity/v3/users/f4ecdf8ab7d84d35adb866f94aaf3eff => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:12.271915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371f29d-15fe-4f07-81f5-e71eb00f9186 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:12.282791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371f29d-15fe-4f07-81f5-e71eb00f9186 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:12.338436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:12.339055 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:12.339339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:12.339627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:12.341473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:12.408683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:12.410195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:12.421524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1272/2530] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:12.454895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:12.457127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:12.575287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58174e-bcd4-4aa3-9fc3-80eca163313b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:12.584357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:12.585042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:12.585332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:12.585611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] PATH_INFO: `/v3/projects/179b9f65028648e18eeb74963c50122a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:12.586761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=179b9f65028648e18eeb74963c50122a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:12.590929 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:12.679612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179b9f65028648e18eeb74963c50122a', 'request_id': 'req-1aac6e4d-bf28-488c-bae6-91e27ddf5442'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.691471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179b9f65028648e18eeb74963c50122a', 'request_id': 'req-1aac6e4d-bf28-488c-bae6-91e27ddf5442'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.708673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179b9f65028648e18eeb74963c50122a', 'request_id': 'req-1aac6e4d-bf28-488c-bae6-91e27ddf5442'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.739546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] The token cache is being invalidate because project 179b9f65028648e18eeb74963c50122a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:12.740371 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.745829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1aac6e4d-bf28-488c-bae6-91e27ddf5442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:12.748608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1259/2531] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:12 2022] DELETE /identity/v3/projects/179b9f65028648e18eeb74963c50122a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:12.845933 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:12.848269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:12.896291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33ab75-44cb-47da-a80a-efc9116044ff service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:13.010742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:13.038435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:13.039164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:13.039519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:13.039873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] PATH_INFO: `/v3/users/6889363e2cae47ada752e74aef71f126` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:13.041335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6889363e2cae47ada752e74aef71f126)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:13.045002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:13.127069 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6889363e2cae47ada752e74aef71f126'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:13.127771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6889363e2cae47ada752e74aef71f126'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:13.138557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6889363e2cae47ada752e74aef71f126'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:13.139491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6889363e2cae47ada752e74aef71f126'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:13.155713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6889363e2cae47ada752e74aef71f126'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:13.193434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1273/2532] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:12 2022] DELETE /identity/v3/users/6889363e2cae47ada752e74aef71f126 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:13.198019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018123-1533-4e3c-9037-0d6b875672e3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:13.199568 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018123-1533-4e3c-9037-0d6b875672e3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:13.199914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018123-1533-4e3c-9037-0d6b875672e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:13.238479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-03018123-1533-4e3c-9037-0d6b875672e3 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:13.239787 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-03018123-1533-4e3c-9037-0d6b875672e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:13.327177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:13.338065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1260/2533] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:13.350592 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:13.354267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:13.436293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af891142-c7f2-468f-99b0-cc21741ed2f7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:13.440724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af475-1928-4b12-b0e0-faf51ed23fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:13.441391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af475-1928-4b12-b0e0-faf51ed23fbf service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:13.441680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af475-1928-4b12-b0e0-faf51ed23fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:13.441952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af475-1928-4b12-b0e0-faf51ed23fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:13.454249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af475-1928-4b12-b0e0-faf51ed23fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:13.456441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af475-1928-4b12-b0e0-faf51ed23fbf service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:13.477115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1274/2534] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:14.023150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03018123-1533-4e3c-9037-0d6b875672e3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:14.025398 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-03018123-1533-4e3c-9037-0d6b875672e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:14.039508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03018123-1533-4e3c-9037-0d6b875672e3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:14.045433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:14.046102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:14.046487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:14.046644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] PATH_INFO: `/v3/projects/d103c5ff00a6461d914e628fa6f094be` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:14.048444 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d103c5ff00a6461d914e628fa6f094be)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:14.050838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:14.113638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd103c5ff00a6461d914e628fa6f094be', 'request_id': 'req-16cf0f92-569f-48ed-82a8-79dc92019d67'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.121251 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd103c5ff00a6461d914e628fa6f094be', 'request_id': 'req-16cf0f92-569f-48ed-82a8-79dc92019d67'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.134994 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd103c5ff00a6461d914e628fa6f094be', 'request_id': 'req-16cf0f92-569f-48ed-82a8-79dc92019d67'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.148911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] The token cache is being invalidate because project d103c5ff00a6461d914e628fa6f094be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:14.149207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.149918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.151537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1261/2535] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:14 2022] DELETE /identity/v3/projects/d103c5ff00a6461d914e628fa6f094be => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:14.155630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af475-1928-4b12-b0e0-faf51ed23fbf service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:14.176154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af475-1928-4b12-b0e0-faf51ed23fbf service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:14.250905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:14.251776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:14.252164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:14.252573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] PATH_INFO: `/v3/users/fd6012ffaf5b4a0da4926eba0f7d5ab7` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:14.255491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6012ffaf5b4a0da4926eba0f7d5ab7)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:14.257814 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:14.272209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:14.357820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6012ffaf5b4a0da4926eba0f7d5ab7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.358760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6012ffaf5b4a0da4926eba0f7d5ab7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.371195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6012ffaf5b4a0da4926eba0f7d5ab7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.372101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6012ffaf5b4a0da4926eba0f7d5ab7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.383270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6012ffaf5b4a0da4926eba0f7d5ab7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:14.407775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1275/2536] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:14 2022] DELETE /identity/v3/users/fd6012ffaf5b4a0da4926eba0f7d5ab7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:15.217580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:15.219003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:15.234300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:15.236424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:15.255789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf0f92-569f-48ed-82a8-79dc92019d67 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:15.372544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:15.374123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:15.375037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:15.375504 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] PATH_INFO: `/v3/users/1eeafdaa1db4436abac229a6f48543a9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:15.377218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eeafdaa1db4436abac229a6f48543a9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:15.378984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae585394-373d-42fd-b95d-4003ffd625f4 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:15.379633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:15.385578 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:15.386465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:15.387381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:15.387739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] PATH_INFO: `/v3/projects/aba9af6e9e1e420ba17ce00bb85e68bc` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:15.389998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba9af6e9e1e420ba17ce00bb85e68bc)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:15.392508 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:15.457090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eeafdaa1db4436abac229a6f48543a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.484901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eeafdaa1db4436abac229a6f48543a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.485932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eeafdaa1db4436abac229a6f48543a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.496611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eeafdaa1db4436abac229a6f48543a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.497443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7d83ac09-9f13-4593-8c9c-ef6ce6e2aeaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eeafdaa1db4436abac229a6f48543a9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.505155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba9af6e9e1e420ba17ce00bb85e68bc', 'request_id': 'req-b7b20702-2d8a-415f-b281-f468ba4d1aae'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.511577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba9af6e9e1e420ba17ce00bb85e68bc', 'request_id': 'req-b7b20702-2d8a-415f-b281-f468ba4d1aae'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.522830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba9af6e9e1e420ba17ce00bb85e68bc', 'request_id': 'req-b7b20702-2d8a-415f-b281-f468ba4d1aae'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.528843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1262/2537] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:15 2022] DELETE /identity/v3/users/1eeafdaa1db4436abac229a6f48543a9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:15.570747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] The token cache is being invalidate because project aba9af6e9e1e420ba17ce00bb85e68bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:15.571001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.572853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-b7b20702-2d8a-415f-b281-f468ba4d1aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.574446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1276/2538] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:15 2022] DELETE /identity/v3/projects/aba9af6e9e1e420ba17ce00bb85e68bc => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:15.579445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a524413-f70a-43cd-8d06-0493c185611a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:15.579740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a524413-f70a-43cd-8d06-0493c185611a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:15.580022 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a524413-f70a-43cd-8d06-0493c185611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:15.592135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:15.619190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8a524413-f70a-43cd-8d06-0493c185611a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:15.624751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8a524413-f70a-43cd-8d06-0493c185611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:15.736091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1263/2539] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:15.737802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a573f33-a448-429f-af4e-2b9c69572d79 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:15.738281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a573f33-a448-429f-af4e-2b9c69572d79 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:15.738562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a573f33-a448-429f-af4e-2b9c69572d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:15.766745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9a573f33-a448-429f-af4e-2b9c69572d79 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:15.768574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9a573f33-a448-429f-af4e-2b9c69572d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:15.780849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1277/2540] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:15.784697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a524413-f70a-43cd-8d06-0493c185611a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:15.792564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a573f33-a448-429f-af4e-2b9c69572d79 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:15.795976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9a573f33-a448-429f-af4e-2b9c69572d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:15.799822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a524413-f70a-43cd-8d06-0493c185611a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:15.813366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d74898-d5ff-4fca-9a91-c5dd6ad6e03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:15.814011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d74898-d5ff-4fca-9a91-c5dd6ad6e03b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:15.814298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d74898-d5ff-4fca-9a91-c5dd6ad6e03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:15.814585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d74898-d5ff-4fca-9a91-c5dd6ad6e03b admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:15.815246 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d74898-d5ff-4fca-9a91-c5dd6ad6e03b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:15.815560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d74898-d5ff-4fca-9a91-c5dd6ad6e03b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:15.817232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d74898-d5ff-4fca-9a91-c5dd6ad6e03b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:15.827399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a573f33-a448-429f-af4e-2b9c69572d79 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:15.831642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1264/2541] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:15.835976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d74898-d5ff-4fca-9a91-c5dd6ad6e03b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:15.837510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:15.838694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:15.839364 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:15.839967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] PATH_INFO: `/v3/projects/65c8194f4a0a45e1888412dc0999e547` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:15.842667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c8194f4a0a45e1888412dc0999e547)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:15.845113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:15.846374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d74898-d5ff-4fca-9a91-c5dd6ad6e03b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:15.851712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6202d-d14b-4a13-bb49-d3e163710229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:15.852332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6202d-d14b-4a13-bb49-d3e163710229 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:15.852618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6202d-d14b-4a13-bb49-d3e163710229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:15.853539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6202d-d14b-4a13-bb49-d3e163710229 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:15.855071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6202d-d14b-4a13-bb49-d3e163710229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:15.855515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:15.856832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6202d-d14b-4a13-bb49-d3e163710229 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:15.903462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1265/2542] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:15 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:15.905320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6202d-d14b-4a13-bb49-d3e163710229 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:15.925966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c8194f4a0a45e1888412dc0999e547', 'request_id': 'req-6b0b68ec-b02d-435f-9282-a00b1f38d983'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.938039 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c8194f4a0a45e1888412dc0999e547', 'request_id': 'req-6b0b68ec-b02d-435f-9282-a00b1f38d983'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.952074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c8194f4a0a45e1888412dc0999e547', 'request_id': 'req-6b0b68ec-b02d-435f-9282-a00b1f38d983'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.976896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] The token cache is being invalidate because project 65c8194f4a0a45e1888412dc0999e547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:15.977201 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.977925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6b0b68ec-b02d-435f-9282-a00b1f38d983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:15.979707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1278/2543] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:15 2022] DELETE /identity/v3/projects/65c8194f4a0a45e1888412dc0999e547 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:15.986742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca67c8-358d-446d-b891-ffc806e48e97 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:15.986742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca67c8-358d-446d-b891-ffc806e48e97 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:15.986989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca67c8-358d-446d-b891-ffc806e48e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:15.995493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6202d-d14b-4a13-bb49-d3e163710229 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.004385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dc81f-4434-4da9-bc45-b1c6ba27ff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.005385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dc81f-4434-4da9-bc45-b1c6ba27ff08 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.005858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dc81f-4434-4da9-bc45-b1c6ba27ff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.006255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dc81f-4434-4da9-bc45-b1c6ba27ff08 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.007039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dc81f-4434-4da9-bc45-b1c6ba27ff08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.010614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dc81f-4434-4da9-bc45-b1c6ba27ff08 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.022174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e5ca67c8-358d-446d-b891-ffc806e48e97 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:16.023404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e5ca67c8-358d-446d-b891-ffc806e48e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.037902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1279/2544] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:16.038865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca67c8-358d-446d-b891-ffc806e48e97 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.047027 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e5ca67c8-358d-446d-b891-ffc806e48e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.050277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1266/2545] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:15 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:16.051482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee1cff-cbaa-4aac-9718-69b51778fbd7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.051765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee1cff-cbaa-4aac-9718-69b51778fbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.052035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee1cff-cbaa-4aac-9718-69b51778fbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.068141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca67c8-358d-446d-b891-ffc806e48e97 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.076104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.076104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.076104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.076104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] PATH_INFO: `/v3/users/e1fdbf0fb6b14090bf38fac073d01814` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.078051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fdbf0fb6b14090bf38fac073d01814)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.081488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.087724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6cee1cff-cbaa-4aac-9718-69b51778fbd7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:16.089253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6cee1cff-cbaa-4aac-9718-69b51778fbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.100979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1267/2546] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:16.100979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee1cff-cbaa-4aac-9718-69b51778fbd7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.102645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6cee1cff-cbaa-4aac-9718-69b51778fbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.115702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee1cff-cbaa-4aac-9718-69b51778fbd7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.121346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46b4ef-4caa-4299-80af-8c6f179b5a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.121949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46b4ef-4caa-4299-80af-8c6f179b5a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.122709 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46b4ef-4caa-4299-80af-8c6f179b5a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.122980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46b4ef-4caa-4299-80af-8c6f179b5a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.123618 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46b4ef-4caa-4299-80af-8c6f179b5a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.125277 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46b4ef-4caa-4299-80af-8c6f179b5a5c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.134826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fdbf0fb6b14090bf38fac073d01814'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.135591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fdbf0fb6b14090bf38fac073d01814'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.136096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1268/2547] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:16.136913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46b4ef-4caa-4299-80af-8c6f179b5a5c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.146384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fdbf0fb6b14090bf38fac073d01814'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.147052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fdbf0fb6b14090bf38fac073d01814'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.147232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46b4ef-4caa-4299-80af-8c6f179b5a5c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.152136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.152726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.153040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.153315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.154045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:16.154315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.154747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fdbf0fb6b14090bf38fac073d01814'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.155856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.168778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1269/2548] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:16.169627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.171355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.172658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1280/2549] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:16 2022] DELETE /identity/v3/users/e1fdbf0fb6b14090bf38fac073d01814 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:16.175216 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.175216 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.188432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d4d33-d59f-449e-bd41-e225883c2175 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.217064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:16.250064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1de676-2efd-4b12-a669-5e91857ec778 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.256494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534c628-5ae8-4c32-a6f9-28aff5c1daa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.256973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288fc7f-489a-40be-94fb-548791633004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.257414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534c628-5ae8-4c32-a6f9-28aff5c1daa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.257414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534c628-5ae8-4c32-a6f9-28aff5c1daa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.257675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288fc7f-489a-40be-94fb-548791633004 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.257789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534c628-5ae8-4c32-a6f9-28aff5c1daa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.257866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288fc7f-489a-40be-94fb-548791633004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.258113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288fc7f-489a-40be-94fb-548791633004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.258306 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534c628-5ae8-4c32-a6f9-28aff5c1daa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.258750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288fc7f-489a-40be-94fb-548791633004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.260471 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534c628-5ae8-4c32-a6f9-28aff5c1daa8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.260647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288fc7f-489a-40be-94fb-548791633004 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.265724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:16.271372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1270/2550] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:16.276116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288fc7f-489a-40be-94fb-548791633004 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.287384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288fc7f-489a-40be-94fb-548791633004 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.293121 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d94e9-a5e9-43d4-8897-b4489e19737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.293792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d94e9-a5e9-43d4-8897-b4489e19737f admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.294084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d94e9-a5e9-43d4-8897-b4489e19737f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.294361 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d94e9-a5e9-43d4-8897-b4489e19737f admin admin] PATH_INFO: `/v3/projects/149e3477c3a24dbfa79880e728ea8737/users/efaed26081034f82b9743415dedc5b54/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.302671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1281/2551] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/projects => generated 401 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:16.307158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534c628-5ae8-4c32-a6f9-28aff5c1daa8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.310745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d94e9-a5e9-43d4-8897-b4489e19737f admin admin] RBAC: Authorizing `identity:create_grant(project_id=149e3477c3a24dbfa79880e728ea8737, user_id=efaed26081034f82b9743415dedc5b54, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.313490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d94e9-a5e9-43d4-8897-b4489e19737f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.329611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1271/2552] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:16 2022] PUT /identity/v3/projects/149e3477c3a24dbfa79880e728ea8737/users/efaed26081034f82b9743415dedc5b54/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:16.341582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a31fd0-1151-4f4a-b4a4-95484d2fa5d2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.342276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a31fd0-1151-4f4a-b4a4-95484d2fa5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.342276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a31fd0-1151-4f4a-b4a4-95484d2fa5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.375904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-32a31fd0-1151-4f4a-b4a4-95484d2fa5d2 None None] MFA Rules not processed for user `efaed26081034f82b9743415dedc5b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:16.377495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-32a31fd0-1151-4f4a-b4a4-95484d2fa5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.383761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534c628-5ae8-4c32-a6f9-28aff5c1daa8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.446306 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1272/2553] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:16.448184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a31fd0-1151-4f4a-b4a4-95484d2fa5d2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.449835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-32a31fd0-1151-4f4a-b4a4-95484d2fa5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.456445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321ed5e-bab3-41c1-bfc3-b79e15ebd660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.457110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321ed5e-bab3-41c1-bfc3-b79e15ebd660 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.457391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321ed5e-bab3-41c1-bfc3-b79e15ebd660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.457661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321ed5e-bab3-41c1-bfc3-b79e15ebd660 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.458295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321ed5e-bab3-41c1-bfc3-b79e15ebd660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.460387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321ed5e-bab3-41c1-bfc3-b79e15ebd660 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.483425 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1282/2554] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/users => generated 301 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:16.484431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321ed5e-bab3-41c1-bfc3-b79e15ebd660 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.494484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321ed5e-bab3-41c1-bfc3-b79e15ebd660 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.499380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca3e78-5844-4666-9dba-235797e89a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.499981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca3e78-5844-4666-9dba-235797e89a59 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.500259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca3e78-5844-4666-9dba-235797e89a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.500531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca3e78-5844-4666-9dba-235797e89a59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.501188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca3e78-5844-4666-9dba-235797e89a59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.503099 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca3e78-5844-4666-9dba-235797e89a59 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.518947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a31fd0-1151-4f4a-b4a4-95484d2fa5d2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.524491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.526321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.526615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.526894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] PATH_INFO: `/v3/projects/5a4cf92cfae0440297266ef8e1d501fc` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.531561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1283/2555] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:16.532389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca3e78-5844-4666-9dba-235797e89a59 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.538584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4cf92cfae0440297266ef8e1d501fc)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.540955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.563807 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:16.589806 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4cf92cfae0440297266ef8e1d501fc', 'request_id': 'req-6262805c-163d-4c49-bfc2-b0e50e5984c1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.595399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4cf92cfae0440297266ef8e1d501fc', 'request_id': 'req-6262805c-163d-4c49-bfc2-b0e50e5984c1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.602498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4cf92cfae0440297266ef8e1d501fc', 'request_id': 'req-6262805c-163d-4c49-bfc2-b0e50e5984c1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.604208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca3e78-5844-4666-9dba-235797e89a59 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.607871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:16.614773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] The token cache is being invalidate because project 5a4cf92cfae0440297266ef8e1d501fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:16.615066 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.615728 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:16.617287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1273/2556] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:16 2022] DELETE /identity/v3/projects/5a4cf92cfae0440297266ef8e1d501fc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:16.618167 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.619640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.667810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda1dcd-298d-4284-b203-b5a2ad1d64b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.668420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda1dcd-298d-4284-b203-b5a2ad1d64b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.668698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda1dcd-298d-4284-b203-b5a2ad1d64b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.669013 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda1dcd-298d-4284-b203-b5a2ad1d64b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.669650 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda1dcd-298d-4284-b203-b5a2ad1d64b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.671315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda1dcd-298d-4284-b203-b5a2ad1d64b1 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.682215 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1284/2557] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:16.683230 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c32b3-9548-419c-8cf1-1cf964d7b017 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.683512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c32b3-9548-419c-8cf1-1cf964d7b017 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.683789 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c32b3-9548-419c-8cf1-1cf964d7b017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.689097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262805c-163d-4c49-bfc2-b0e50e5984c1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.694939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c56b6-fe3a-4bb0-89a8-f833fa20f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.695529 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c56b6-fe3a-4bb0-89a8-f833fa20f988 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.695940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c56b6-fe3a-4bb0-89a8-f833fa20f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.696317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c56b6-fe3a-4bb0-89a8-f833fa20f988 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.697029 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c56b6-fe3a-4bb0-89a8-f833fa20f988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.699424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c56b6-fe3a-4bb0-89a8-f833fa20f988 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.703005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-226c32b3-9548-419c-8cf1-1cf964d7b017 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:16.704213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-226c32b3-9548-419c-8cf1-1cf964d7b017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.713166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1285/2558] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:16.714369 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c32b3-9548-419c-8cf1-1cf964d7b017 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.715721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-226c32b3-9548-419c-8cf1-1cf964d7b017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.722830 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1274/2559] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:16.724392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c22ed-9105-4198-a795-7b88323f2ede None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.724681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c22ed-9105-4198-a795-7b88323f2ede None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.724986 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c22ed-9105-4198-a795-7b88323f2ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.730781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c32b3-9548-419c-8cf1-1cf964d7b017 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.736465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d9342-295e-45fc-8a5c-ca736308ce22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.737240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9342-295e-45fc-8a5c-ca736308ce22 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.737572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9342-295e-45fc-8a5c-ca736308ce22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.737916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9342-295e-45fc-8a5c-ca736308ce22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.738643 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d9342-295e-45fc-8a5c-ca736308ce22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.740626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d9342-295e-45fc-8a5c-ca736308ce22 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.746783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4a3c22ed-9105-4198-a795-7b88323f2ede None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:16.748042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4a3c22ed-9105-4198-a795-7b88323f2ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.755220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1286/2560] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:16.756130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d9342-295e-45fc-8a5c-ca736308ce22 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.765465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1275/2561] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:16.766630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d9342-295e-45fc-8a5c-ca736308ce22 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.769392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c22ed-9105-4198-a795-7b88323f2ede None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.771716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec38fe-96ef-4a42-b2f5-ac422305d15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.772322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec38fe-96ef-4a42-b2f5-ac422305d15a admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.772606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec38fe-96ef-4a42-b2f5-ac422305d15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.772956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec38fe-96ef-4a42-b2f5-ac422305d15a admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.773621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec38fe-96ef-4a42-b2f5-ac422305d15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.776093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec38fe-96ef-4a42-b2f5-ac422305d15a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.791199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1287/2562] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:16.792011 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec38fe-96ef-4a42-b2f5-ac422305d15a admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.798025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c22ed-9105-4198-a795-7b88323f2ede None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.802948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e748b3-d063-45f3-9942-4d70c2e58220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.803584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e748b3-d063-45f3-9942-4d70c2e58220 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.803867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e748b3-d063-45f3-9942-4d70c2e58220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.804917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e748b3-d063-45f3-9942-4d70c2e58220 admin admin] PATH_INFO: `/v3/projects/73272a4c8bcc42039c33155d5f3ba060/users/2f14cbcdae41428ab95e6d69b4441286/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.805404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec38fe-96ef-4a42-b2f5-ac422305d15a admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.810655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a9cd5-e151-4fd9-a844-71a3601f4294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.811311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a9cd5-e151-4fd9-a844-71a3601f4294 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.811602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a9cd5-e151-4fd9-a844-71a3601f4294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.811879 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a9cd5-e151-4fd9-a844-71a3601f4294 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.812538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a9cd5-e151-4fd9-a844-71a3601f4294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:16.812876 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a9cd5-e151-4fd9-a844-71a3601f4294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.814496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a9cd5-e151-4fd9-a844-71a3601f4294 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.822785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e748b3-d063-45f3-9942-4d70c2e58220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73272a4c8bcc42039c33155d5f3ba060, user_id=2f14cbcdae41428ab95e6d69b4441286, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.825733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e748b3-d063-45f3-9942-4d70c2e58220 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.830543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1288/2563] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:16.830543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a9cd5-e151-4fd9-a844-71a3601f4294 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.840377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a9cd5-e151-4fd9-a844-71a3601f4294 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.845392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e381c63-ff6b-4c24-bcdf-6bd8f0713d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.846081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e381c63-ff6b-4c24-bcdf-6bd8f0713d1a admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.846458 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e381c63-ff6b-4c24-bcdf-6bd8f0713d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.846804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e381c63-ff6b-4c24-bcdf-6bd8f0713d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.847559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e381c63-ff6b-4c24-bcdf-6bd8f0713d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.849222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1276/2564] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:16 2022] PUT /identity/v3/projects/73272a4c8bcc42039c33155d5f3ba060/users/2f14cbcdae41428ab95e6d69b4441286/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:16.849336 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e381c63-ff6b-4c24-bcdf-6bd8f0713d1a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.850322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e748b3-d063-45f3-9942-4d70c2e58220 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.861808 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1289/2565] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:16.863192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75f327-e6e1-4ea0-8ff7-6d9967f93d8b None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.863478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75f327-e6e1-4ea0-8ff7-6d9967f93d8b None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.863822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75f327-e6e1-4ea0-8ff7-6d9967f93d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.883416 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3c75f327-e6e1-4ea0-8ff7-6d9967f93d8b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:16.885151 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3c75f327-e6e1-4ea0-8ff7-6d9967f93d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.927087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e748b3-d063-45f3-9942-4d70c2e58220 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.932360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834cdd9-de07-40f6-bb7f-82c8df750348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.933537 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834cdd9-de07-40f6-bb7f-82c8df750348 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.933981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834cdd9-de07-40f6-bb7f-82c8df750348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.934285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834cdd9-de07-40f6-bb7f-82c8df750348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.934958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834cdd9-de07-40f6-bb7f-82c8df750348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.936990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834cdd9-de07-40f6-bb7f-82c8df750348 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:16.951198 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1290/2566] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:16.952139 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:16.952285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75f327-e6e1-4ea0-8ff7-6d9967f93d8b None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.963908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75f327-e6e1-4ea0-8ff7-6d9967f93d8b None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:16.972117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8083013-8556-4ed9-a247-bc1865f80a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:16.973212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8083013-8556-4ed9-a247-bc1865f80a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:16.973494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8083013-8556-4ed9-a247-bc1865f80a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:16.973767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8083013-8556-4ed9-a247-bc1865f80a4d admin admin] PATH_INFO: `/v3/projects/b0f29268a8734772939b60d008585e4d/users/572e9b6612334ee1ac6bda237182c9e5/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:16.977134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1277/2567] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:16 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:16.979366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834cdd9-de07-40f6-bb7f-82c8df750348 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:16.981140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4834cdd9-de07-40f6-bb7f-82c8df750348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:16.992522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8083013-8556-4ed9-a247-bc1865f80a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f29268a8734772939b60d008585e4d, user_id=572e9b6612334ee1ac6bda237182c9e5, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:16.996000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8083013-8556-4ed9-a247-bc1865f80a4d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.011054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1291/2568] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:16 2022] PUT /identity/v3/projects/b0f29268a8734772939b60d008585e4d/users/572e9b6612334ee1ac6bda237182c9e5/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:17.013021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8083013-8556-4ed9-a247-bc1865f80a4d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.048289 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:17.053785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834cdd9-de07-40f6-bb7f-82c8df750348 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.093328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8083013-8556-4ed9-a247-bc1865f80a4d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.099290 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea20b1f-160d-419e-af08-342b5e5d2583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.099912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea20b1f-160d-419e-af08-342b5e5d2583 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.100179 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea20b1f-160d-419e-af08-342b5e5d2583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.100454 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea20b1f-160d-419e-af08-342b5e5d2583 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.101120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea20b1f-160d-419e-af08-342b5e5d2583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.104662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea20b1f-160d-419e-af08-342b5e5d2583 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.117694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b8e15-e897-4def-98c5-5355aa08d638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.118390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b8e15-e897-4def-98c5-5355aa08d638 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.118701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b8e15-e897-4def-98c5-5355aa08d638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.119073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b8e15-e897-4def-98c5-5355aa08d638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.129423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1292/2569] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:17 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:17.130666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaea94d-765b-41c4-b8e4-cd895fd13843 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.130947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaea94d-765b-41c4-b8e4-cd895fd13843 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.131217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaea94d-765b-41c4-b8e4-cd895fd13843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.142493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b8e15-e897-4def-98c5-5355aa08d638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.144023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b8e15-e897-4def-98c5-5355aa08d638 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.155428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1278/2570] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:17.156267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b8e15-e897-4def-98c5-5355aa08d638 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.167065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b8e15-e897-4def-98c5-5355aa08d638 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.172689 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61249180-79f1-4fa9-a405-94e99876bdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.173340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61249180-79f1-4fa9-a405-94e99876bdcc admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.173623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61249180-79f1-4fa9-a405-94e99876bdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.173898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61249180-79f1-4fa9-a405-94e99876bdcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.174612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61249180-79f1-4fa9-a405-94e99876bdcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.176166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61249180-79f1-4fa9-a405-94e99876bdcc admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.178514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3aaea94d-765b-41c4-b8e4-cd895fd13843 None None] MFA Rules not processed for user `572e9b6612334ee1ac6bda237182c9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:17.180134 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3aaea94d-765b-41c4-b8e4-cd895fd13843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:17.192403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1279/2571] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:17.197698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61249180-79f1-4fa9-a405-94e99876bdcc admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.208965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61249180-79f1-4fa9-a405-94e99876bdcc admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.220134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e18b4-4f79-4089-a073-9f3fbf2870e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.221129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e18b4-4f79-4089-a073-9f3fbf2870e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.221243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e18b4-4f79-4089-a073-9f3fbf2870e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.221460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e18b4-4f79-4089-a073-9f3fbf2870e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.222169 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e18b4-4f79-4089-a073-9f3fbf2870e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.224177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e18b4-4f79-4089-a073-9f3fbf2870e9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.236697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1280/2572] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:17.240976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e18b4-4f79-4089-a073-9f3fbf2870e9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.252864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e18b4-4f79-4089-a073-9f3fbf2870e9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.258406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d9f8e-c2e5-41b2-a581-ce3d8e33a56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.259696 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9f8e-c2e5-41b2-a581-ce3d8e33a56e admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.260135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9f8e-c2e5-41b2-a581-ce3d8e33a56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.260499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9f8e-c2e5-41b2-a581-ce3d8e33a56e admin admin] PATH_INFO: `/v3/projects/3719b545a9f24c94a17358ba1978c8a8/users/4872797e3e6f44b189c333c89b9acd91/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.265078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1293/2573] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:17.269710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaea94d-765b-41c4-b8e4-cd895fd13843 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.278680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d9f8e-c2e5-41b2-a581-ce3d8e33a56e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3719b545a9f24c94a17358ba1978c8a8, user_id=4872797e3e6f44b189c333c89b9acd91, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.281797 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d9f8e-c2e5-41b2-a581-ce3d8e33a56e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.285127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaea94d-765b-41c4-b8e4-cd895fd13843 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.291144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e95187-caad-443c-a807-d953d9955e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.292048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e95187-caad-443c-a807-d953d9955e73 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.292415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e95187-caad-443c-a807-d953d9955e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.292838 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e95187-caad-443c-a807-d953d9955e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.293451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e95187-caad-443c-a807-d953d9955e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.295551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e95187-caad-443c-a807-d953d9955e73 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.298159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1281/2574] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:17 2022] PUT /identity/v3/projects/3719b545a9f24c94a17358ba1978c8a8/users/4872797e3e6f44b189c333c89b9acd91/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:17.311387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741bd4d-3cf1-46a1-be9f-0ff5389146ee None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.312077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741bd4d-3cf1-46a1-be9f-0ff5389146ee None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.313360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741bd4d-3cf1-46a1-be9f-0ff5389146ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.343945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1294/2575] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:17 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:17.345115 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e95187-caad-443c-a807-d953d9955e73 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.367729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6741bd4d-3cf1-46a1-be9f-0ff5389146ee None None] MFA Rules not processed for user `4872797e3e6f44b189c333c89b9acd91`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:17.369083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6741bd4d-3cf1-46a1-be9f-0ff5389146ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:17.436508 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e95187-caad-443c-a807-d953d9955e73 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.442504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069679aa-b173-4b04-8c32-98a90bc87c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.443506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069679aa-b173-4b04-8c32-98a90bc87c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.443674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069679aa-b173-4b04-8c32-98a90bc87c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.443972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069679aa-b173-4b04-8c32-98a90bc87c49 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.445126 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069679aa-b173-4b04-8c32-98a90bc87c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.447810 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069679aa-b173-4b04-8c32-98a90bc87c49 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.475633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1295/2576] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:17 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:17.479792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069679aa-b173-4b04-8c32-98a90bc87c49 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.480508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:17.500500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1282/2577] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:17.501798 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741bd4d-3cf1-46a1-be9f-0ff5389146ee None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.502142 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069679aa-b173-4b04-8c32-98a90bc87c49 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.517437 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741bd4d-3cf1-46a1-be9f-0ff5389146ee None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.517951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85878fbd-0416-4292-a821-faff55489f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.519596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85878fbd-0416-4292-a821-faff55489f38 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.520089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85878fbd-0416-4292-a821-faff55489f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.520934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85878fbd-0416-4292-a821-faff55489f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.521783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85878fbd-0416-4292-a821-faff55489f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.524572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85878fbd-0416-4292-a821-faff55489f38 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.526643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002973e-4d2e-4fad-aa88-859d7822b27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.527740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002973e-4d2e-4fad-aa88-859d7822b27a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.527740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002973e-4d2e-4fad-aa88-859d7822b27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.529077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002973e-4d2e-4fad-aa88-859d7822b27a admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.529854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002973e-4d2e-4fad-aa88-859d7822b27a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.532271 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002973e-4d2e-4fad-aa88-859d7822b27a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.542144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1296/2578] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:17.543074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85878fbd-0416-4292-a821-faff55489f38 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.548671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:17.559777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85878fbd-0416-4292-a821-faff55489f38 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.561702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ff626-1efc-4630-8a45-bec65db2444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.564205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ff626-1efc-4630-8a45-bec65db2444d admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.564831 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ff626-1efc-4630-8a45-bec65db2444d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.565193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ff626-1efc-4630-8a45-bec65db2444d admin admin] PATH_INFO: `/v3/projects/f371969888fb45e3b23ae36cb9233be1/users/cb878c434a0e4866841942d0ada16449/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.577882 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1283/2579] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:17 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:17.581731 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002973e-4d2e-4fad-aa88-859d7822b27a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.589927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ff626-1efc-4630-8a45-bec65db2444d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f371969888fb45e3b23ae36cb9233be1, user_id=cb878c434a0e4866841942d0ada16449, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.593507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ff626-1efc-4630-8a45-bec65db2444d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.612852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1297/2580] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:17 2022] PUT /identity/v3/projects/f371969888fb45e3b23ae36cb9233be1/users/cb878c434a0e4866841942d0ada16449/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:17.617097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ff626-1efc-4630-8a45-bec65db2444d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.677213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002973e-4d2e-4fad-aa88-859d7822b27a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.715089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ff626-1efc-4630-8a45-bec65db2444d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.722577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f5952-9d3a-45b1-9335-0a88093ac295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.723511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f5952-9d3a-45b1-9335-0a88093ac295 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.723710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f5952-9d3a-45b1-9335-0a88093ac295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.724113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f5952-9d3a-45b1-9335-0a88093ac295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.724792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f5952-9d3a-45b1-9335-0a88093ac295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.726759 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f5952-9d3a-45b1-9335-0a88093ac295 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.741728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1298/2581] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:17.747339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f5952-9d3a-45b1-9335-0a88093ac295 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.753663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2718aad-1805-41e7-8811-29e1cfa7ba0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.754377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2718aad-1805-41e7-8811-29e1cfa7ba0b admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.754729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2718aad-1805-41e7-8811-29e1cfa7ba0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.755321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2718aad-1805-41e7-8811-29e1cfa7ba0b admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.756046 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2718aad-1805-41e7-8811-29e1cfa7ba0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.758801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2718aad-1805-41e7-8811-29e1cfa7ba0b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.762258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f5952-9d3a-45b1-9335-0a88093ac295 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.767841 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da247f30-7fbe-4d0e-a875-82383f0ce1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.768582 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da247f30-7fbe-4d0e-a875-82383f0ce1ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.768982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da247f30-7fbe-4d0e-a875-82383f0ce1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.769357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da247f30-7fbe-4d0e-a875-82383f0ce1ca admin admin] PATH_INFO: `/v3/projects/f371969888fb45e3b23ae36cb9233be1/users/cb878c434a0e4866841942d0ada16449/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.771547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da247f30-7fbe-4d0e-a875-82383f0ce1ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f371969888fb45e3b23ae36cb9233be1, user_id=cb878c434a0e4866841942d0ada16449, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.775262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da247f30-7fbe-4d0e-a875-82383f0ce1ca admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.791634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1284/2582] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:17 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:17.793269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2718aad-1805-41e7-8811-29e1cfa7ba0b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.796172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1299/2583] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:17 2022] PUT /identity/v3/projects/f371969888fb45e3b23ae36cb9233be1/users/cb878c434a0e4866841942d0ada16449/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:17.808250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2718aad-1805-41e7-8811-29e1cfa7ba0b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.809169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fde81-9a18-4fb4-ba6b-8661c0bccb73 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.809462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fde81-9a18-4fb4-ba6b-8661c0bccb73 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.809794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fde81-9a18-4fb4-ba6b-8661c0bccb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.862982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-845fde81-9a18-4fb4-ba6b-8661c0bccb73 None None] MFA Rules not processed for user `cb878c434a0e4866841942d0ada16449`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:17.864402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-845fde81-9a18-4fb4-ba6b-8661c0bccb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:17.892659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826c25d-110c-45e3-9187-c7d3f83f82ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.893942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826c25d-110c-45e3-9187-c7d3f83f82ab admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.894415 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826c25d-110c-45e3-9187-c7d3f83f82ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.894888 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826c25d-110c-45e3-9187-c7d3f83f82ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.896267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826c25d-110c-45e3-9187-c7d3f83f82ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.900407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826c25d-110c-45e3-9187-c7d3f83f82ab admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:17.923055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1285/2584] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:17.927112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826c25d-110c-45e3-9187-c7d3f83f82ab admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.945506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826c25d-110c-45e3-9187-c7d3f83f82ab admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:17.954399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1300/2585] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:17.958225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd917f99-4b77-4416-aade-2f7983f0b628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:17.959995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd917f99-4b77-4416-aade-2f7983f0b628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:17.960516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd917f99-4b77-4416-aade-2f7983f0b628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:17.961129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd917f99-4b77-4416-aade-2f7983f0b628 admin admin] PATH_INFO: `/v3/projects/d9a16cd53a834402b71f2fe0ad7483ee/users/a300e8de452e4646b70df03b0f234de8/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:17.962815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fde81-9a18-4fb4-ba6b-8661c0bccb73 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:17.964513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-845fde81-9a18-4fb4-ba6b-8661c0bccb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:17.964708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:17.989004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd917f99-4b77-4416-aade-2f7983f0b628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a16cd53a834402b71f2fe0ad7483ee, user_id=a300e8de452e4646b70df03b0f234de8, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:17.991949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd917f99-4b77-4416-aade-2f7983f0b628 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.007533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1286/2586] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:17 2022] PUT /identity/v3/projects/d9a16cd53a834402b71f2fe0ad7483ee/users/a300e8de452e4646b70df03b0f234de8/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:18.012493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd917f99-4b77-4416-aade-2f7983f0b628 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:18.028268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:18.033981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fde81-9a18-4fb4-ba6b-8661c0bccb73 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.093077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd917f99-4b77-4416-aade-2f7983f0b628 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.098694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33faf0-4f0f-4aec-99b1-e89365090220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.099312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33faf0-4f0f-4aec-99b1-e89365090220 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.099595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33faf0-4f0f-4aec-99b1-e89365090220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.099869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33faf0-4f0f-4aec-99b1-e89365090220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.100508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33faf0-4f0f-4aec-99b1-e89365090220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.102161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33faf0-4f0f-4aec-99b1-e89365090220 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.104307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac834b-e8e6-4b5d-829f-f89870a322fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.105235 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac834b-e8e6-4b5d-829f-f89870a322fc service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.105653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac834b-e8e6-4b5d-829f-f89870a322fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.106035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac834b-e8e6-4b5d-829f-f89870a322fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.108455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-45ac834b-e8e6-4b5d-829f-f89870a322fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:18.113873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1287/2587] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:18.117829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33faf0-4f0f-4aec-99b1-e89365090220 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:18.128955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33faf0-4f0f-4aec-99b1-e89365090220 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.135133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cadfc-ad3c-4a7e-ad55-638fd4f387ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.135797 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cadfc-ad3c-4a7e-ad55-638fd4f387ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.136088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cadfc-ad3c-4a7e-ad55-638fd4f387ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.136460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cadfc-ad3c-4a7e-ad55-638fd4f387ad admin admin] PATH_INFO: `/v3/projects/d9a16cd53a834402b71f2fe0ad7483ee/users/a300e8de452e4646b70df03b0f234de8/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.139544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2cadfc-ad3c-4a7e-ad55-638fd4f387ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a16cd53a834402b71f2fe0ad7483ee, user_id=a300e8de452e4646b70df03b0f234de8, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.142274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2cadfc-ad3c-4a7e-ad55-638fd4f387ad admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.157644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1288/2588] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:18 2022] PUT /identity/v3/projects/d9a16cd53a834402b71f2fe0ad7483ee/users/a300e8de452e4646b70df03b0f234de8/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:18.167432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c5348-7f45-48bb-a289-39ea2b161533 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.167741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c5348-7f45-48bb-a289-39ea2b161533 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.168075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c5348-7f45-48bb-a289-39ea2b161533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.201467 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b41c5348-7f45-48bb-a289-39ea2b161533 None None] MFA Rules not processed for user `a300e8de452e4646b70df03b0f234de8`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:18.203019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b41c5348-7f45-48bb-a289-39ea2b161533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:18.205435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac834b-e8e6-4b5d-829f-f89870a322fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.207664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac834b-e8e6-4b5d-829f-f89870a322fc service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.227964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1301/2589] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:18.248839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00df801-d3cd-46ca-8eec-b7cd38899da9 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.248839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00df801-d3cd-46ca-8eec-b7cd38899da9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.248839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00df801-d3cd-46ca-8eec-b7cd38899da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.296508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1289/2590] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:18.302300 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d00df801-d3cd-46ca-8eec-b7cd38899da9 None None] MFA Rules not processed for user `2f14cbcdae41428ab95e6d69b4441286`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:18.304223 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d00df801-d3cd-46ca-8eec-b7cd38899da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:18.335839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c5348-7f45-48bb-a289-39ea2b161533 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:18.337592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b41c5348-7f45-48bb-a289-39ea2b161533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:18.422957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1302/2591] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:18 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:18.428419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00df801-d3cd-46ca-8eec-b7cd38899da9 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:18.444257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c5348-7f45-48bb-a289-39ea2b161533 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.450388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a0d61-61b7-4b6c-bd16-f52aae94e236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.451322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a0d61-61b7-4b6c-bd16-f52aae94e236 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.451811 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a0d61-61b7-4b6c-bd16-f52aae94e236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.452691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a0d61-61b7-4b6c-bd16-f52aae94e236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.461322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:18.470719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a0d61-61b7-4b6c-bd16-f52aae94e236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.473187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a0d61-61b7-4b6c-bd16-f52aae94e236 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.499577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1290/2592] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:18.513383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:18.533208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00df801-d3cd-46ca-8eec-b7cd38899da9 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.544469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cf5c7-a758-473e-ae95-fda997c04987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.545371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cf5c7-a758-473e-ae95-fda997c04987 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.545603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cf5c7-a758-473e-ae95-fda997c04987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.545855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cf5c7-a758-473e-ae95-fda997c04987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.546508 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6cf5c7-a758-473e-ae95-fda997c04987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.549378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6cf5c7-a758-473e-ae95-fda997c04987 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.591193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1303/2593] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:18 2022] POST /identity/v3/projects => generated 405 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:18.595019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a0d61-61b7-4b6c-bd16-f52aae94e236 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:18.609052 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a0d61-61b7-4b6c-bd16-f52aae94e236 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.691892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee7c6b-01e3-4184-ad72-1b5cdd366d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.693375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee7c6b-01e3-4184-ad72-1b5cdd366d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.693866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee7c6b-01e3-4184-ad72-1b5cdd366d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.694223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee7c6b-01e3-4184-ad72-1b5cdd366d93 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.694881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee7c6b-01e3-4184-ad72-1b5cdd366d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.698329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee7c6b-01e3-4184-ad72-1b5cdd366d93 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.724636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1291/2594] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:18 2022] POST /identity/v3/users => generated 309 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:18.729181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cf5c7-a758-473e-ae95-fda997c04987 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:18.742299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cf5c7-a758-473e-ae95-fda997c04987 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.749414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d50d6c-4cfe-43d3-9e48-feeb8bbf9089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.750396 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d50d6c-4cfe-43d3-9e48-feeb8bbf9089 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.750698 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d50d6c-4cfe-43d3-9e48-feeb8bbf9089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.750974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d50d6c-4cfe-43d3-9e48-feeb8bbf9089 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.751627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d50d6c-4cfe-43d3-9e48-feeb8bbf9089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.753429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d50d6c-4cfe-43d3-9e48-feeb8bbf9089 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.769380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1304/2595] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:18.770491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee7c6b-01e3-4184-ad72-1b5cdd366d93 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:18.783181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee7c6b-01e3-4184-ad72-1b5cdd366d93 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.789380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6113e-b45e-4682-9112-3bce34ebbe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.790209 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6113e-b45e-4682-9112-3bce34ebbe3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.790857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6113e-b45e-4682-9112-3bce34ebbe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.791220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6113e-b45e-4682-9112-3bce34ebbe3d admin admin] PATH_INFO: `/v3/projects/e70297b825a4472e8d6d4bce3cd6d2d0/users/8f1829e6fb194263959fba1bb51e40bf/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.810723 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6113e-b45e-4682-9112-3bce34ebbe3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70297b825a4472e8d6d4bce3cd6d2d0, user_id=8f1829e6fb194263959fba1bb51e40bf, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.814040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6113e-b45e-4682-9112-3bce34ebbe3d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.828107 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1292/2596] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:18 2022] PUT /identity/v3/projects/e70297b825a4472e8d6d4bce3cd6d2d0/users/8f1829e6fb194263959fba1bb51e40bf/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:18.831909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d50d6c-4cfe-43d3-9e48-feeb8bbf9089 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:18.844417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d50d6c-4cfe-43d3-9e48-feeb8bbf9089 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.940320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b4d40-f76c-45aa-b65a-602fdfc7bc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.940973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b4d40-f76c-45aa-b65a-602fdfc7bc1c admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.941255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b4d40-f76c-45aa-b65a-602fdfc7bc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.941527 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b4d40-f76c-45aa-b65a-602fdfc7bc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.942169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b4d40-f76c-45aa-b65a-602fdfc7bc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.944109 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b4d40-f76c-45aa-b65a-602fdfc7bc1c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:18.957931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1305/2597] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:18.963048 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6113e-b45e-4682-9112-3bce34ebbe3d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:18.975336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6113e-b45e-4682-9112-3bce34ebbe3d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:18.980606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7b62b-557d-4d1e-a664-03e51c1164e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:18.981350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7b62b-557d-4d1e-a664-03e51c1164e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:18.981632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7b62b-557d-4d1e-a664-03e51c1164e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:18.981905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7b62b-557d-4d1e-a664-03e51c1164e6 admin admin] PATH_INFO: `/v3/projects/e70297b825a4472e8d6d4bce3cd6d2d0/users/8f1829e6fb194263959fba1bb51e40bf/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:18.983981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7b62b-557d-4d1e-a664-03e51c1164e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70297b825a4472e8d6d4bce3cd6d2d0, user_id=8f1829e6fb194263959fba1bb51e40bf, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:18.986674 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7b62b-557d-4d1e-a664-03e51c1164e6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:19.000122 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1293/2598] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:18 2022] PUT /identity/v3/projects/e70297b825a4472e8d6d4bce3cd6d2d0/users/8f1829e6fb194263959fba1bb51e40bf/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:19.123646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b4d40-f76c-45aa-b65a-602fdfc7bc1c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:19.134645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b4d40-f76c-45aa-b65a-602fdfc7bc1c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:19.203847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:19.204651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:19.204970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:19.205241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] PATH_INFO: `/v3/users/4872797e3e6f44b189c333c89b9acd91` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:19.206382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] RBAC: Authorizing `identity:delete_user(user_id=4872797e3e6f44b189c333c89b9acd91)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:19.208337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:19.257219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4872797e3e6f44b189c333c89b9acd91'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:19.257934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4872797e3e6f44b189c333c89b9acd91'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:19.266526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4872797e3e6f44b189c333c89b9acd91'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:19.267248 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4872797e3e6f44b189c333c89b9acd91'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:19.271943 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:19.275657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4872797e3e6f44b189c333c89b9acd91'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:19.296273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1306/2599] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:19 2022] DELETE /identity/v3/users/4872797e3e6f44b189c333c89b9acd91 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:19.301656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee1553-7af6-4a56-9132-d00a94993152 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:19.302207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee1553-7af6-4a56-9132-d00a94993152 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:19.302305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee1553-7af6-4a56-9132-d00a94993152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:19.347124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-04ee1553-7af6-4a56-9132-d00a94993152 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:19.348392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-04ee1553-7af6-4a56-9132-d00a94993152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:19.424291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1294/2600] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:19.435305 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:19.436769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:19.497694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a0bc3-80d9-4793-b77d-8a940be0b9be admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:19.501920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868eea6-2cfb-465a-990f-d4ed55237681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:19.502535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868eea6-2cfb-465a-990f-d4ed55237681 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:19.502855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868eea6-2cfb-465a-990f-d4ed55237681 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:19.503132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868eea6-2cfb-465a-990f-d4ed55237681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:19.512985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868eea6-2cfb-465a-990f-d4ed55237681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:19.514438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868eea6-2cfb-465a-990f-d4ed55237681 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:19.525547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1307/2601] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:19.660220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a1453-5fe2-4976-9d74-b836a920a0bc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:19.661138 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a1453-5fe2-4976-9d74-b836a920a0bc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:19.661646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a1453-5fe2-4976-9d74-b836a920a0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:19.707907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7a7a1453-5fe2-4976-9d74-b836a920a0bc None None] MFA Rules not processed for user `8f1829e6fb194263959fba1bb51e40bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:19.709256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7a7a1453-5fe2-4976-9d74-b836a920a0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:19.764760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:19.814337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1295/2602] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:19 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:19.867403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868eea6-2cfb-465a-990f-d4ed55237681 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:19.869197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3868eea6-2cfb-465a-990f-d4ed55237681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:19.897519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868eea6-2cfb-465a-990f-d4ed55237681 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:19.907054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:19.907973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:19.908441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:19.909096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] PATH_INFO: `/v3/projects/3719b545a9f24c94a17358ba1978c8a8` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:19.938148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3719b545a9f24c94a17358ba1978c8a8)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:19.940192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:20.012419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3719b545a9f24c94a17358ba1978c8a8', 'request_id': 'req-23dee8f8-6a21-43de-8096-3a91f3ae7f89'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.018615 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a1453-5fe2-4976-9d74-b836a920a0bc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:20.018793 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3719b545a9f24c94a17358ba1978c8a8', 'request_id': 'req-23dee8f8-6a21-43de-8096-3a91f3ae7f89'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.020030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7a7a1453-5fe2-4976-9d74-b836a920a0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:20.028012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3719b545a9f24c94a17358ba1978c8a8', 'request_id': 'req-23dee8f8-6a21-43de-8096-3a91f3ae7f89'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.040576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] The token cache is being invalidate because project 3719b545a9f24c94a17358ba1978c8a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:20.040922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.041602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.043333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1308/2603] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:19 2022] DELETE /identity/v3/projects/3719b545a9f24c94a17358ba1978c8a8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:20.047213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:20.052367 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:20.104000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a1453-5fe2-4976-9d74-b836a920a0bc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:20.162878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dee8f8-6a21-43de-8096-3a91f3ae7f89 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:20.171830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:20.172488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:20.172772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:20.173068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f9cd9-96ed-480f-9c8a-b27e159d8303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:20.173186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] PATH_INFO: `/v3/users/a300e8de452e4646b70df03b0f234de8` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:20.174003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f9cd9-96ed-480f-9c8a-b27e159d8303 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:20.174387 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a300e8de452e4646b70df03b0f234de8)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:20.174879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f9cd9-96ed-480f-9c8a-b27e159d8303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:20.175377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f9cd9-96ed-480f-9c8a-b27e159d8303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:20.176363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:20.178195 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c96f9cd9-96ed-480f-9c8a-b27e159d8303 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:20.180181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:20.261541 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a300e8de452e4646b70df03b0f234de8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.262263 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a300e8de452e4646b70df03b0f234de8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.267909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f9cd9-96ed-480f-9c8a-b27e159d8303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:20.269693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f9cd9-96ed-480f-9c8a-b27e159d8303 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:20.273094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a300e8de452e4646b70df03b0f234de8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.273902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a300e8de452e4646b70df03b0f234de8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.281701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1296/2604] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:20.293437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ec0b2b76-7c8b-4e69-946c-817c1bdf52ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a300e8de452e4646b70df03b0f234de8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.317546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1309/2605] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:20 2022] DELETE /identity/v3/users/a300e8de452e4646b70df03b0f234de8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:20.700644 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f9cd9-96ed-480f-9c8a-b27e159d8303 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:20.720081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f9cd9-96ed-480f-9c8a-b27e159d8303 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:20.778878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:20.807541 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:20.808340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:20.808716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:20.809140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] PATH_INFO: `/v3/projects/d9a16cd53a834402b71f2fe0ad7483ee` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:20.810449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a16cd53a834402b71f2fe0ad7483ee)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:20.812846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:20.876043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a16cd53a834402b71f2fe0ad7483ee', 'request_id': 'req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.885201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a16cd53a834402b71f2fe0ad7483ee', 'request_id': 'req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.896253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a16cd53a834402b71f2fe0ad7483ee', 'request_id': 'req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.911718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] The token cache is being invalidate because project d9a16cd53a834402b71f2fe0ad7483ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:20.912203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.913090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-43b09a0c-ad7a-4ad7-b810-68b39ebb5edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:20.915736 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1297/2606] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:20 2022] DELETE /identity/v3/projects/d9a16cd53a834402b71f2fe0ad7483ee => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:20.921498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36d543-c91d-4862-bb23-01aa9460ede5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:20.921994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36d543-c91d-4862-bb23-01aa9460ede5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:20.921994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36d543-c91d-4862-bb23-01aa9460ede5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:20.947462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-6a36d543-c91d-4862-bb23-01aa9460ede5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:20.949499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6a36d543-c91d-4862-bb23-01aa9460ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.019991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1310/2607] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:21.027922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ae01e-9912-419b-8304-1040249114bf None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.028225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ae01e-9912-419b-8304-1040249114bf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.028507 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ae01e-9912-419b-8304-1040249114bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.052855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e56ae01e-9912-419b-8304-1040249114bf None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:21.052997 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36d543-c91d-4862-bb23-01aa9460ede5 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.054050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e56ae01e-9912-419b-8304-1040249114bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.054333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6a36d543-c91d-4862-bb23-01aa9460ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.066979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1298/2608] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:21.068153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ae01e-9912-419b-8304-1040249114bf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.083848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ae01e-9912-419b-8304-1040249114bf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.089133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.089756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.090036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.090315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.090965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:21.091268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.092911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.109099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1299/2609] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:21.109099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.110940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.135385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36d543-c91d-4862-bb23-01aa9460ede5 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.142001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2921a80-ba85-42b1-ac19-764cc3842efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.142672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2921a80-ba85-42b1-ac19-764cc3842efe service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.142952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2921a80-ba85-42b1-ac19-764cc3842efe service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.143228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2921a80-ba85-42b1-ac19-764cc3842efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.145124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f2921a80-ba85-42b1-ac19-764cc3842efe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.182199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:21.199157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3591e50-79e3-4dd9-8201-6bae11ef6bde admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.205230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.205832 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.206105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.206374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.208065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.215680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2921a80-ba85-42b1-ac19-764cc3842efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.217174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2921a80-ba85-42b1-ac19-764cc3842efe service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.219905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.221526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.226989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1311/2610] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:21.228152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2921a80-ba85-42b1-ac19-764cc3842efe service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.232428 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1300/2611] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:21.233327 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.234863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.240170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2921a80-ba85-42b1-ac19-764cc3842efe service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.247760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3b694-4f01-492d-830a-b6e29380f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.248378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3b694-4f01-492d-830a-b6e29380f7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.248788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be337744-305b-4131-9f4c-e4db39363260 service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.248922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3b694-4f01-492d-830a-b6e29380f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.249002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3b694-4f01-492d-830a-b6e29380f7c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.249916 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3b694-4f01-492d-830a-b6e29380f7c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.252025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3b694-4f01-492d-830a-b6e29380f7c1 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.259284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.260099 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.260535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.260853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] PATH_INFO: `/v3/users/2f14cbcdae41428ab95e6d69b4441286` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.261989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f14cbcdae41428ab95e6d69b4441286)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.264020 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.291718 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1312/2612] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:21 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:21.295805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3b694-4f01-492d-830a-b6e29380f7c1 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.317706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:21.329501 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f14cbcdae41428ab95e6d69b4441286'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.339096 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f14cbcdae41428ab95e6d69b4441286'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.339769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f14cbcdae41428ab95e6d69b4441286'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.347035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f14cbcdae41428ab95e6d69b4441286'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.353053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f14cbcdae41428ab95e6d69b4441286'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.372546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1301/2613] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:21 2022] DELETE /identity/v3/users/2f14cbcdae41428ab95e6d69b4441286 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:21.377561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3b694-4f01-492d-830a-b6e29380f7c1 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.438037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df33e63-eb19-40ee-9008-70883c71870f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.438734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df33e63-eb19-40ee-9008-70883c71870f admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.439021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df33e63-eb19-40ee-9008-70883c71870f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.439341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df33e63-eb19-40ee-9008-70883c71870f admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.440541 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df33e63-eb19-40ee-9008-70883c71870f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.442641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df33e63-eb19-40ee-9008-70883c71870f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.470911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1313/2614] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:21 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:21.472478 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.474297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.492702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae03b01-a6c6-4115-b670-62262eabf49f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.500933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092ab98-df17-48ae-8d07-cfc349399ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.501724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092ab98-df17-48ae-8d07-cfc349399ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.502011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092ab98-df17-48ae-8d07-cfc349399ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.502324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092ab98-df17-48ae-8d07-cfc349399ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.503177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092ab98-df17-48ae-8d07-cfc349399ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.504866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092ab98-df17-48ae-8d07-cfc349399ea2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.521258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df33e63-eb19-40ee-9008-70883c71870f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.527499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1302/2615] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:21.543729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df33e63-eb19-40ee-9008-70883c71870f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.549934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b98967-304d-42f4-a426-a7d7a850d4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.550558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b98967-304d-42f4-a426-a7d7a850d4ae admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.550837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b98967-304d-42f4-a426-a7d7a850d4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.551107 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b98967-304d-42f4-a426-a7d7a850d4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.551780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b98967-304d-42f4-a426-a7d7a850d4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.553461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b98967-304d-42f4-a426-a7d7a850d4ae admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.568054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1314/2616] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:21.573411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092ab98-df17-48ae-8d07-cfc349399ea2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.584321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092ab98-df17-48ae-8d07-cfc349399ea2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.584551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b98967-304d-42f4-a426-a7d7a850d4ae admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.586571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-88b98967-304d-42f4-a426-a7d7a850d4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.589580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e309a-b0f2-4155-b8d6-100775c2b6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.590318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e309a-b0f2-4155-b8d6-100775c2b6d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.590600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e309a-b0f2-4155-b8d6-100775c2b6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.590873 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e309a-b0f2-4155-b8d6-100775c2b6d0 admin admin] PATH_INFO: `/v3/projects/02fe4cddc7e647aa92f281966568f94f/users/2d989829230c475c9d2f2f4f280d637b/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.603854 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b98967-304d-42f4-a426-a7d7a850d4ae admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.609357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e309a-b0f2-4155-b8d6-100775c2b6d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02fe4cddc7e647aa92f281966568f94f, user_id=2d989829230c475c9d2f2f4f280d637b, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.612237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e309a-b0f2-4155-b8d6-100775c2b6d0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.612317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.613007 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.613280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.613648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] PATH_INFO: `/v3/projects/73272a4c8bcc42039c33155d5f3ba060` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.614853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] RBAC: Authorizing `identity:delete_project(project_id=73272a4c8bcc42039c33155d5f3ba060)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.616924 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.626042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1303/2617] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:21 2022] PUT /identity/v3/projects/02fe4cddc7e647aa92f281966568f94f/users/2d989829230c475c9d2f2f4f280d637b/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:21.638708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8173-41ad-4412-ba79-5a263f98451f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.639105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8173-41ad-4412-ba79-5a263f98451f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.639454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8173-41ad-4412-ba79-5a263f98451f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.654543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:21.670343 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3a5e8173-41ad-4412-ba79-5a263f98451f None None] MFA Rules not processed for user `2d989829230c475c9d2f2f4f280d637b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:21.671253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73272a4c8bcc42039c33155d5f3ba060', 'request_id': 'req-14e0297c-a058-4bee-aec7-9e496c6ae57f'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.671647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3a5e8173-41ad-4412-ba79-5a263f98451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.677515 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73272a4c8bcc42039c33155d5f3ba060', 'request_id': 'req-14e0297c-a058-4bee-aec7-9e496c6ae57f'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.686908 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73272a4c8bcc42039c33155d5f3ba060', 'request_id': 'req-14e0297c-a058-4bee-aec7-9e496c6ae57f'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.699678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] The token cache is being invalidate because project 73272a4c8bcc42039c33155d5f3ba060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:21.699967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.700632 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.702150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1315/2618] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:21 2022] DELETE /identity/v3/projects/73272a4c8bcc42039c33155d5f3ba060 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:21.706023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.707408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.739605 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1304/2619] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:21.744642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8173-41ad-4412-ba79-5a263f98451f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.746067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3a5e8173-41ad-4412-ba79-5a263f98451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.773073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0297c-a058-4bee-aec7-9e496c6ae57f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.777961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.778591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.778883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.779154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] PATH_INFO: `/v3/users/8f1829e6fb194263959fba1bb51e40bf` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.780288 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f1829e6fb194263959fba1bb51e40bf)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.782239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.819984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8173-41ad-4412-ba79-5a263f98451f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.825738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e23ef-fe6c-469f-8c6d-ce9af542ac8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.826356 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e23ef-fe6c-469f-8c6d-ce9af542ac8d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.826646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e23ef-fe6c-469f-8c6d-ce9af542ac8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.826920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e23ef-fe6c-469f-8c6d-ce9af542ac8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.827558 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e23ef-fe6c-469f-8c6d-ce9af542ac8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.829538 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e23ef-fe6c-469f-8c6d-ce9af542ac8d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:21.831485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f1829e6fb194263959fba1bb51e40bf'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.832146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f1829e6fb194263959fba1bb51e40bf'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.839645 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f1829e6fb194263959fba1bb51e40bf'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.840281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f1829e6fb194263959fba1bb51e40bf'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.844911 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:21.847872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f1829e6fb194263959fba1bb51e40bf'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:21.861517 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1305/2620] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:21 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:21.865677 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e23ef-fe6c-469f-8c6d-ce9af542ac8d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:21.867302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-254e23ef-fe6c-469f-8c6d-ce9af542ac8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:21.867466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1316/2621] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:21 2022] DELETE /identity/v3/users/8f1829e6fb194263959fba1bb51e40bf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:21.933836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e23ef-fe6c-469f-8c6d-ce9af542ac8d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:21.993639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93521eb-aca1-41d1-914a-13f1f2bd46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:21.994712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93521eb-aca1-41d1-914a-13f1f2bd46b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:21.994995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93521eb-aca1-41d1-914a-13f1f2bd46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:21.995268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93521eb-aca1-41d1-914a-13f1f2bd46b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:21.995899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93521eb-aca1-41d1-914a-13f1f2bd46b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:21.997885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93521eb-aca1-41d1-914a-13f1f2bd46b3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.021920 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1306/2622] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:21 2022] POST /identity/v3/users => generated 300 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:22.026967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.038052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe2d5d-79d4-4173-adb2-377c3ee5ce02 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.042636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93521eb-aca1-41d1-914a-13f1f2bd46b3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.044869 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e93521eb-aca1-41d1-914a-13f1f2bd46b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.045836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62320ed9-d4ac-4425-a224-b52b62cb11a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.046690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320ed9-d4ac-4425-a224-b52b62cb11a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.046961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320ed9-d4ac-4425-a224-b52b62cb11a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.047235 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320ed9-d4ac-4425-a224-b52b62cb11a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.047886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62320ed9-d4ac-4425-a224-b52b62cb11a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.049653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62320ed9-d4ac-4425-a224-b52b62cb11a8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.061302 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93521eb-aca1-41d1-914a-13f1f2bd46b3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.061657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1317/2623] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:22.065817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62320ed9-d4ac-4425-a224-b52b62cb11a8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.067671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.076547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.076917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.077197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] PATH_INFO: `/v3/projects/e70297b825a4472e8d6d4bce3cd6d2d0` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.077311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62320ed9-d4ac-4425-a224-b52b62cb11a8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.078599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70297b825a4472e8d6d4bce3cd6d2d0)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.080626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.082566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367a883-44e0-46b9-9b93-dc2b054a75f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.083591 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a883-44e0-46b9-9b93-dc2b054a75f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.084483 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a883-44e0-46b9-9b93-dc2b054a75f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.084773 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a883-44e0-46b9-9b93-dc2b054a75f2 admin admin] PATH_INFO: `/v3/projects/d9c1c2659da44a998c10d299b3fd5fde/users/adfad691c2254955a984ef565296e4bf/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.100324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367a883-44e0-46b9-9b93-dc2b054a75f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c1c2659da44a998c10d299b3fd5fde, user_id=adfad691c2254955a984ef565296e4bf, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.103189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367a883-44e0-46b9-9b93-dc2b054a75f2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.116180 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1318/2624] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:22 2022] PUT /identity/v3/projects/d9c1c2659da44a998c10d299b3fd5fde/users/adfad691c2254955a984ef565296e4bf/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:22.120546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367a883-44e0-46b9-9b93-dc2b054a75f2 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.124940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:22.134878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70297b825a4472e8d6d4bce3cd6d2d0', 'request_id': 'req-8df8ad1d-8975-4922-aee5-65a524a505f3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.141394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70297b825a4472e8d6d4bce3cd6d2d0', 'request_id': 'req-8df8ad1d-8975-4922-aee5-65a524a505f3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.148819 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70297b825a4472e8d6d4bce3cd6d2d0', 'request_id': 'req-8df8ad1d-8975-4922-aee5-65a524a505f3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.189631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367a883-44e0-46b9-9b93-dc2b054a75f2 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.252958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d278d5-080c-41ee-9632-a8e3b712b403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.253495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d278d5-080c-41ee-9632-a8e3b712b403 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.253775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d278d5-080c-41ee-9632-a8e3b712b403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.254048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d278d5-080c-41ee-9632-a8e3b712b403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.254695 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d278d5-080c-41ee-9632-a8e3b712b403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.256322 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d278d5-080c-41ee-9632-a8e3b712b403 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.270686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1319/2625] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:22.271915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d278d5-080c-41ee-9632-a8e3b712b403 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.282393 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d278d5-080c-41ee-9632-a8e3b712b403 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.285024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] The token cache is being invalidate because project e70297b825a4472e8d6d4bce3cd6d2d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:22.285319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.285989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8df8ad1d-8975-4922-aee5-65a524a505f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.287293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69f274-05d7-410f-83cf-006175fc4004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.287487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1307/2626] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:22 2022] DELETE /identity/v3/projects/e70297b825a4472e8d6d4bce3cd6d2d0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:22.287938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69f274-05d7-410f-83cf-006175fc4004 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.288292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69f274-05d7-410f-83cf-006175fc4004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.288566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69f274-05d7-410f-83cf-006175fc4004 admin admin] PATH_INFO: `/v3/projects/d9c1c2659da44a998c10d299b3fd5fde/users/adfad691c2254955a984ef565296e4bf/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.291127 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69f274-05d7-410f-83cf-006175fc4004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c1c2659da44a998c10d299b3fd5fde, user_id=adfad691c2254955a984ef565296e4bf, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.292486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f2bb-6b0e-4ab0-b3f8-e28f08008191 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.292777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f2bb-6b0e-4ab0-b3f8-e28f08008191 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.293081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f2bb-6b0e-4ab0-b3f8-e28f08008191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.293818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69f274-05d7-410f-83cf-006175fc4004 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.304893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1320/2627] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:22 2022] PUT /identity/v3/projects/d9c1c2659da44a998c10d299b3fd5fde/users/adfad691c2254955a984ef565296e4bf/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:22.315103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fa050-61ca-4d26-8910-1197ca6ff1cc None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.315394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fa050-61ca-4d26-8910-1197ca6ff1cc None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.315670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fa050-61ca-4d26-8910-1197ca6ff1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.315948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6049f2bb-6b0e-4ab0-b3f8-e28f08008191 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:22.317234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6049f2bb-6b0e-4ab0-b3f8-e28f08008191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.326831 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:22.345413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-391fa050-61ca-4d26-8910-1197ca6ff1cc None None] MFA Rules not processed for user `adfad691c2254955a984ef565296e4bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:22.346648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-391fa050-61ca-4d26-8910-1197ca6ff1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.393536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1308/2628] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:22.427054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1321/2629] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:22.440267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f2bb-6b0e-4ab0-b3f8-e28f08008191 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.441734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6049f2bb-6b0e-4ab0-b3f8-e28f08008191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.454338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf0496-064b-44a8-8b92-970139dcaf69 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.454652 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf0496-064b-44a8-8b92-970139dcaf69 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.454963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf0496-064b-44a8-8b92-970139dcaf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.479737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d0bf0496-064b-44a8-8b92-970139dcaf69 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:22.481927 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d0bf0496-064b-44a8-8b92-970139dcaf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.489415 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1322/2630] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:22.493857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf0496-064b-44a8-8b92-970139dcaf69 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.503358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf0496-064b-44a8-8b92-970139dcaf69 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.504658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f2bb-6b0e-4ab0-b3f8-e28f08008191 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.508074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860c8ea-9fb9-4c7c-bd41-f81c1ef11900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.508340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7d9a7-8ee1-4c41-aeb2-c60173168497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.509260 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860c8ea-9fb9-4c7c-bd41-f81c1ef11900 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.509506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7d9a7-8ee1-4c41-aeb2-c60173168497 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.509611 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860c8ea-9fb9-4c7c-bd41-f81c1ef11900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.509788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7d9a7-8ee1-4c41-aeb2-c60173168497 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.509955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860c8ea-9fb9-4c7c-bd41-f81c1ef11900 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.510055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7d9a7-8ee1-4c41-aeb2-c60173168497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.510600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860c8ea-9fb9-4c7c-bd41-f81c1ef11900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:22.510941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860c8ea-9fb9-4c7c-bd41-f81c1ef11900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.512505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860c8ea-9fb9-4c7c-bd41-f81c1ef11900 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.519212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7d9a7-8ee1-4c41-aeb2-c60173168497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.520545 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:22.520688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7d9a7-8ee1-4c41-aeb2-c60173168497 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.527501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1323/2631] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:22.532086 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860c8ea-9fb9-4c7c-bd41-f81c1ef11900 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.533895 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1309/2632] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:22.544499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860c8ea-9fb9-4c7c-bd41-f81c1ef11900 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.551461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eba87-0aaa-4429-8997-a3e7df46712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.552080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eba87-0aaa-4429-8997-a3e7df46712a admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.552505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eba87-0aaa-4429-8997-a3e7df46712a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.552666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eba87-0aaa-4429-8997-a3e7df46712a admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.553285 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eba87-0aaa-4429-8997-a3e7df46712a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.555232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eba87-0aaa-4429-8997-a3e7df46712a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.597138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1324/2633] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:22 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:22.600619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7d9a7-8ee1-4c41-aeb2-c60173168497 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.612909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7d9a7-8ee1-4c41-aeb2-c60173168497 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.677504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eba87-0aaa-4429-8997-a3e7df46712a admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.679310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-398eba87-0aaa-4429-8997-a3e7df46712a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.693713 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5113f0-34ea-4aff-a781-2dd58303c8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.694784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5113f0-34ea-4aff-a781-2dd58303c8ed admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.695811 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5113f0-34ea-4aff-a781-2dd58303c8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.696828 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5113f0-34ea-4aff-a781-2dd58303c8ed admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.697885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5113f0-34ea-4aff-a781-2dd58303c8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.700305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5113f0-34ea-4aff-a781-2dd58303c8ed admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.734390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1310/2634] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:22 2022] POST /identity/v3/users => generated 283 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:22.736161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5113f0-34ea-4aff-a781-2dd58303c8ed admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.747901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5113f0-34ea-4aff-a781-2dd58303c8ed admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.754823 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0119f6a-d06c-416c-87ed-208c6cfff716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.755380 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0119f6a-d06c-416c-87ed-208c6cfff716 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.755661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0119f6a-d06c-416c-87ed-208c6cfff716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.755931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0119f6a-d06c-416c-87ed-208c6cfff716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.756681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0119f6a-d06c-416c-87ed-208c6cfff716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.758497 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0119f6a-d06c-416c-87ed-208c6cfff716 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.765376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eba87-0aaa-4429-8997-a3e7df46712a admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.771139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.771545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1311/2635] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:22.771770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.772152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.772548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] PATH_INFO: `/v3/users/2d989829230c475c9d2f2f4f280d637b` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.773792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d989829230c475c9d2f2f4f280d637b)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.775699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0119f6a-d06c-416c-87ed-208c6cfff716 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.776140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.786639 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0119f6a-d06c-416c-87ed-208c6cfff716 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.791460 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c597ea2-0acf-4d76-8bdf-a4a8ab55ad61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.792051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c597ea2-0acf-4d76-8bdf-a4a8ab55ad61 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.792332 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c597ea2-0acf-4d76-8bdf-a4a8ab55ad61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.792628 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c597ea2-0acf-4d76-8bdf-a4a8ab55ad61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.793275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c597ea2-0acf-4d76-8bdf-a4a8ab55ad61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.794857 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c597ea2-0acf-4d76-8bdf-a4a8ab55ad61 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.805962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1312/2636] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:22.811246 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c597ea2-0acf-4d76-8bdf-a4a8ab55ad61 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.822954 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c597ea2-0acf-4d76-8bdf-a4a8ab55ad61 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.827956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0f3e2-b27f-4a0e-a8f3-f89b8a514c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.828702 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0f3e2-b27f-4a0e-a8f3-f89b8a514c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.829105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0f3e2-b27f-4a0e-a8f3-f89b8a514c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.829453 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0f3e2-b27f-4a0e-a8f3-f89b8a514c08 admin admin] PATH_INFO: `/v3/projects/f893d46cab514bde9f2eaf943019872c/users/9d86a1724d324e2180d0f7c8ceacc71f/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.830177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d989829230c475c9d2f2f4f280d637b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.830843 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d989829230c475c9d2f2f4f280d637b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.838270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d989829230c475c9d2f2f4f280d637b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.839097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d989829230c475c9d2f2f4f280d637b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.845193 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0f3e2-b27f-4a0e-a8f3-f89b8a514c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f893d46cab514bde9f2eaf943019872c, user_id=9d86a1724d324e2180d0f7c8ceacc71f, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:22.846587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-05eff4fc-2573-4afb-a1f3-a2cef737cb99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d989829230c475c9d2f2f4f280d637b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:22.847835 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0f3e2-b27f-4a0e-a8f3-f89b8a514c08 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:22.856514 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:22.864729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1313/2637] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:22 2022] PUT /identity/v3/projects/f893d46cab514bde9f2eaf943019872c/users/9d86a1724d324e2180d0f7c8ceacc71f/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:22.865963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1325/2638] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:22 2022] DELETE /identity/v3/users/2d989829230c475c9d2f2f4f280d637b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:22.870977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db2758-43f4-48a9-b9db-57f0925b6321 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.871270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db2758-43f4-48a9-b9db-57f0925b6321 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.871579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db2758-43f4-48a9-b9db-57f0925b6321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.875390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4b8c8-8b43-48bb-952a-21e82aeb36bc None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:22.875679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4b8c8-8b43-48bb-952a-21e82aeb36bc None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:22.875952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4b8c8-8b43-48bb-952a-21e82aeb36bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:22.892067 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-38db2758-43f4-48a9-b9db-57f0925b6321 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:22.893319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-38db2758-43f4-48a9-b9db-57f0925b6321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.904781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-bab4b8c8-8b43-48bb-952a-21e82aeb36bc None None] MFA Rules not processed for user `9d86a1724d324e2180d0f7c8ceacc71f`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:22.906024 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-bab4b8c8-8b43-48bb-952a-21e82aeb36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.950971 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:22.953094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1314/2639] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:22.963658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db2758-43f4-48a9-b9db-57f0925b6321 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.965104 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-38db2758-43f4-48a9-b9db-57f0925b6321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.974357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1326/2640] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:22.978873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4b8c8-8b43-48bb-952a-21e82aeb36bc None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:22.980199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-bab4b8c8-8b43-48bb-952a-21e82aeb36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:22.993892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4b8c8-8b43-48bb-952a-21e82aeb36bc None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:22.999178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d481f2-b484-47dd-b70a-2241dce602bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:22.999771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d481f2-b484-47dd-b70a-2241dce602bc admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.000053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d481f2-b484-47dd-b70a-2241dce602bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.000320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d481f2-b484-47dd-b70a-2241dce602bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.001420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d481f2-b484-47dd-b70a-2241dce602bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.002923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d481f2-b484-47dd-b70a-2241dce602bc admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.023912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db2758-43f4-48a9-b9db-57f0925b6321 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.028776 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec3bb-6381-45de-bcfa-3b31ed982490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.029405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec3bb-6381-45de-bcfa-3b31ed982490 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.029685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec3bb-6381-45de-bcfa-3b31ed982490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.029955 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec3bb-6381-45de-bcfa-3b31ed982490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.034607 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1327/2641] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:22 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:23.036258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d481f2-b484-47dd-b70a-2241dce602bc admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.039461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aec3bb-6381-45de-bcfa-3b31ed982490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.041055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aec3bb-6381-45de-bcfa-3b31ed982490 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.050487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1315/2642] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:23.101053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d481f2-b484-47dd-b70a-2241dce602bc admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.107218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0eeb2c-9247-40f8-b9e6-c080e4385e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.107967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0eeb2c-9247-40f8-b9e6-c080e4385e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.108246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0eeb2c-9247-40f8-b9e6-c080e4385e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.108517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0eeb2c-9247-40f8-b9e6-c080e4385e58 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.109187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0eeb2c-9247-40f8-b9e6-c080e4385e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.111114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0eeb2c-9247-40f8-b9e6-c080e4385e58 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.135288 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1328/2643] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:23 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:23.139400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec3bb-6381-45de-bcfa-3b31ed982490 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.149075 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec3bb-6381-45de-bcfa-3b31ed982490 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.154516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6f342-6489-4fca-a11f-95559a3f1ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.155200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6f342-6489-4fca-a11f-95559a3f1ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.155489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6f342-6489-4fca-a11f-95559a3f1ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.155768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6f342-6489-4fca-a11f-95559a3f1ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.156448 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6f342-6489-4fca-a11f-95559a3f1ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.158431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6f342-6489-4fca-a11f-95559a3f1ad1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.169303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1316/2644] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:23.173325 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0eeb2c-9247-40f8-b9e6-c080e4385e58 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.184171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0eeb2c-9247-40f8-b9e6-c080e4385e58 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.188899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fa8d9-c4c5-4fa5-a72e-258ff544d951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.189560 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fa8d9-c4c5-4fa5-a72e-258ff544d951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.189842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fa8d9-c4c5-4fa5-a72e-258ff544d951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.190112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fa8d9-c4c5-4fa5-a72e-258ff544d951 admin admin] PATH_INFO: `/v3/projects/e9f7734f7745496ab7885a77a7509612/users/373c51db9c3040038917be942276b1fc/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.210736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947fa8d9-c4c5-4fa5-a72e-258ff544d951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f7734f7745496ab7885a77a7509612, user_id=373c51db9c3040038917be942276b1fc, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.214059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947fa8d9-c4c5-4fa5-a72e-258ff544d951 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.228869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1329/2645] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:23 2022] PUT /identity/v3/projects/e9f7734f7745496ab7885a77a7509612/users/373c51db9c3040038917be942276b1fc/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:23.232949 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6f342-6489-4fca-a11f-95559a3f1ad1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.243513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6f342-6489-4fca-a11f-95559a3f1ad1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.302141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fda5b-1195-46e6-be5e-5c5a99a590f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.302885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fda5b-1195-46e6-be5e-5c5a99a590f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.303186 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fda5b-1195-46e6-be5e-5c5a99a590f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.303438 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fda5b-1195-46e6-be5e-5c5a99a590f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.304070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fda5b-1195-46e6-be5e-5c5a99a590f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.307215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fda5b-1195-46e6-be5e-5c5a99a590f2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.319916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1317/2646] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:23.324885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fa8d9-c4c5-4fa5-a72e-258ff544d951 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.336630 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fa8d9-c4c5-4fa5-a72e-258ff544d951 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.338784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fda5b-1195-46e6-be5e-5c5a99a590f2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.340459 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3d6fda5b-1195-46e6-be5e-5c5a99a590f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:23.342478 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93b310-3b4f-44d1-848b-549469d730af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.343266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93b310-3b4f-44d1-848b-549469d730af admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.343617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93b310-3b4f-44d1-848b-549469d730af admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.343950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93b310-3b4f-44d1-848b-549469d730af admin admin] PATH_INFO: `/v3/projects/e9f7734f7745496ab7885a77a7509612/users/373c51db9c3040038917be942276b1fc/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.346988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93b310-3b4f-44d1-848b-549469d730af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f7734f7745496ab7885a77a7509612, user_id=373c51db9c3040038917be942276b1fc, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.349910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93b310-3b4f-44d1-848b-549469d730af admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.356018 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fda5b-1195-46e6-be5e-5c5a99a590f2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.361081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.361720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.362002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.362274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] PATH_INFO: `/v3/projects/02fe4cddc7e647aa92f281966568f94f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.363411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02fe4cddc7e647aa92f281966568f94f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.364056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1330/2647] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:23 2022] PUT /identity/v3/projects/e9f7734f7745496ab7885a77a7509612/users/373c51db9c3040038917be942276b1fc/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:23.365367 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.375546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe1e8e-4013-45fd-9de3-a108b8ae64f5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.375834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe1e8e-4013-45fd-9de3-a108b8ae64f5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.376105 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe1e8e-4013-45fd-9de3-a108b8ae64f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.396063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:23.423287 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02fe4cddc7e647aa92f281966568f94f', 'request_id': 'req-b738c6a1-1873-4f99-ab41-1a75def33f07'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.429590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02fe4cddc7e647aa92f281966568f94f', 'request_id': 'req-b738c6a1-1873-4f99-ab41-1a75def33f07'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.438710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02fe4cddc7e647aa92f281966568f94f', 'request_id': 'req-b738c6a1-1873-4f99-ab41-1a75def33f07'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.452840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] The token cache is being invalidate because project 02fe4cddc7e647aa92f281966568f94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:23.453168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.453935 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.455965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1318/2648] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:23 2022] DELETE /identity/v3/projects/02fe4cddc7e647aa92f281966568f94f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:23.460165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.462254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:23.529840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c6a1-1873-4f99-ab41-1a75def33f07 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.530060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2fbe1e8e-4013-45fd-9de3-a108b8ae64f5 None None] MFA Rules not processed for user `373c51db9c3040038917be942276b1fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:23.531171 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2fbe1e8e-4013-45fd-9de3-a108b8ae64f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:23.534405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.535033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.535316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.535584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] PATH_INFO: `/v3/users/adfad691c2254955a984ef565296e4bf` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.541334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adfad691c2254955a984ef565296e4bf)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.541334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.586867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adfad691c2254955a984ef565296e4bf'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.594447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adfad691c2254955a984ef565296e4bf'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.595576 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adfad691c2254955a984ef565296e4bf'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.602946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adfad691c2254955a984ef565296e4bf'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.603941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adfad691c2254955a984ef565296e4bf'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:23.604517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:23.608457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1331/2649] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:23.620616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe1e8e-4013-45fd-9de3-a108b8ae64f5 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.622102 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2fbe1e8e-4013-45fd-9de3-a108b8ae64f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:23.623788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1319/2650] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:23 2022] DELETE /identity/v3/users/adfad691c2254955a984ef565296e4bf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:23.694281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe1e8e-4013-45fd-9de3-a108b8ae64f5 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.754907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.755283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.755563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.755857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.757638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:23.837328 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:23.841561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:23.853348 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1332/2651] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:23.928276 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.935282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:23.943794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:23.946896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:23.959130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c810055-3dcc-4610-9429-1b2a40333b24 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:23.965253 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ec2cc-48aa-42e4-9382-225effb1bd0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:23.966498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ec2cc-48aa-42e4-9382-225effb1bd0e service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:23.967186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ec2cc-48aa-42e4-9382-225effb1bd0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:23.967685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ec2cc-48aa-42e4-9382-225effb1bd0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:23.969701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e24ec2cc-48aa-42e4-9382-225effb1bd0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:23.974244 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:24.014015 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db9569-102f-4015-abc2-fc4b3b22f301 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:24.018964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:24.019599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.019880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.020150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] PATH_INFO: `/v3/projects/d9c1c2659da44a998c10d299b3fd5fde` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.021388 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c1c2659da44a998c10d299b3fd5fde)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.023417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.041554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ec2cc-48aa-42e4-9382-225effb1bd0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.042749 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ec2cc-48aa-42e4-9382-225effb1bd0e service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.053177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1333/2652] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:24.080715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c1c2659da44a998c10d299b3fd5fde', 'request_id': 'req-b7a8e910-def0-4d80-b001-7d6d575068e3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.087707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c1c2659da44a998c10d299b3fd5fde', 'request_id': 'req-b7a8e910-def0-4d80-b001-7d6d575068e3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.095797 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c1c2659da44a998c10d299b3fd5fde', 'request_id': 'req-b7a8e910-def0-4d80-b001-7d6d575068e3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.110404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] The token cache is being invalidate because project d9c1c2659da44a998c10d299b3fd5fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:24.110738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.111645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b7a8e910-def0-4d80-b001-7d6d575068e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.113985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1320/2653] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:23 2022] DELETE /identity/v3/projects/d9c1c2659da44a998c10d299b3fd5fde => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:24.119671 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43769f91-d940-4b97-9a52-fbc37e956cfe None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.119963 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43769f91-d940-4b97-9a52-fbc37e956cfe None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.120225 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43769f91-d940-4b97-9a52-fbc37e956cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.140512 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:24.147590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-43769f91-d940-4b97-9a52-fbc37e956cfe None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:24.149173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-43769f91-d940-4b97-9a52-fbc37e956cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:24.229358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1334/2654] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:24.236552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231e057-16f8-450f-a22e-ee778c1eced7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.236915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231e057-16f8-450f-a22e-ee778c1eced7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.237234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231e057-16f8-450f-a22e-ee778c1eced7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.257119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5231e057-16f8-450f-a22e-ee778c1eced7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:24.258411 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5231e057-16f8-450f-a22e-ee778c1eced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:24.266977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1321/2655] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:24.271190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43769f91-d940-4b97-9a52-fbc37e956cfe None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:24.280952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43769f91-d940-4b97-9a52-fbc37e956cfe None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:24.285599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694fb7b-db5a-4fac-bcc2-2246b4420e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:24.286214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694fb7b-db5a-4fac-bcc2-2246b4420e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.286494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694fb7b-db5a-4fac-bcc2-2246b4420e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.286767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694fb7b-db5a-4fac-bcc2-2246b4420e49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.287409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694fb7b-db5a-4fac-bcc2-2246b4420e49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:24.287713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694fb7b-db5a-4fac-bcc2-2246b4420e49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.289350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694fb7b-db5a-4fac-bcc2-2246b4420e49 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.301036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1335/2656] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 11:32:24.306097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231e057-16f8-450f-a22e-ee778c1eced7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:24.316352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231e057-16f8-450f-a22e-ee778c1eced7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:24.321894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719ecd0-552d-45c9-98d8-200c0dc9c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:24.322542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719ecd0-552d-45c9-98d8-200c0dc9c942 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.322833 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719ecd0-552d-45c9-98d8-200c0dc9c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.323087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719ecd0-552d-45c9-98d8-200c0dc9c942 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.323708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719ecd0-552d-45c9-98d8-200c0dc9c942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.325799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719ecd0-552d-45c9-98d8-200c0dc9c942 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.353322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1322/2657] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:24 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:24.357736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694fb7b-db5a-4fac-bcc2-2246b4420e49 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:24.369085 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694fb7b-db5a-4fac-bcc2-2246b4420e49 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:24.427566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d617a94-3af8-42b1-8b1e-2aea3a98ab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:24.428165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617a94-3af8-42b1-8b1e-2aea3a98ab80 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.428444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617a94-3af8-42b1-8b1e-2aea3a98ab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.428712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617a94-3af8-42b1-8b1e-2aea3a98ab80 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.429372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d617a94-3af8-42b1-8b1e-2aea3a98ab80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.432080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d617a94-3af8-42b1-8b1e-2aea3a98ab80 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.454594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1336/2658] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:24 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:24.458724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719ecd0-552d-45c9-98d8-200c0dc9c942 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:24.469135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719ecd0-552d-45c9-98d8-200c0dc9c942 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:24.473662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c3117-9398-4304-9ab4-cbdc4d5bf80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:24.474254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c3117-9398-4304-9ab4-cbdc4d5bf80b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.474532 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c3117-9398-4304-9ab4-cbdc4d5bf80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.474816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c3117-9398-4304-9ab4-cbdc4d5bf80b admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.475435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c3117-9398-4304-9ab4-cbdc4d5bf80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.477065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c3117-9398-4304-9ab4-cbdc4d5bf80b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.487027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1323/2659] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:24.490880 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d617a94-3af8-42b1-8b1e-2aea3a98ab80 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:24.501548 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d617a94-3af8-42b1-8b1e-2aea3a98ab80 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:24.506192 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4d989-0b35-46c8-8038-b304fae5f167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:24.506781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4d989-0b35-46c8-8038-b304fae5f167 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.507058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4d989-0b35-46c8-8038-b304fae5f167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.507329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4d989-0b35-46c8-8038-b304fae5f167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.507966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4d989-0b35-46c8-8038-b304fae5f167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.510012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4d989-0b35-46c8-8038-b304fae5f167 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.519844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1337/2660] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:24.523995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c3117-9398-4304-9ab4-cbdc4d5bf80b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:24.534671 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c3117-9398-4304-9ab4-cbdc4d5bf80b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:24.539387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b1267-8d89-4024-8b0b-568c8d800cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:24.540179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b1267-8d89-4024-8b0b-568c8d800cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.540462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b1267-8d89-4024-8b0b-568c8d800cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.540760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b1267-8d89-4024-8b0b-568c8d800cd0 admin admin] PATH_INFO: `/v3/projects/83ab212541a84c40917cf7cbeaf0790a/users/52b7819c1db848059cb5776e5b6d30b0/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.555971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b1267-8d89-4024-8b0b-568c8d800cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ab212541a84c40917cf7cbeaf0790a, user_id=52b7819c1db848059cb5776e5b6d30b0, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.559177 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b1267-8d89-4024-8b0b-568c8d800cd0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.572781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1324/2661] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:24 2022] PUT /identity/v3/projects/83ab212541a84c40917cf7cbeaf0790a/users/52b7819c1db848059cb5776e5b6d30b0/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:24.584158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94692c-a88b-4318-8382-e376744018e6 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.584448 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94692c-a88b-4318-8382-e376744018e6 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.584725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94692c-a88b-4318-8382-e376744018e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.612056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-fe94692c-a88b-4318-8382-e376744018e6 None None] MFA Rules not processed for user `52b7819c1db848059cb5776e5b6d30b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:24.613316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fe94692c-a88b-4318-8382-e376744018e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:24.673857 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:24.677800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1338/2662] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:24.689652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b1267-8d89-4024-8b0b-568c8d800cd0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:24.691298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-481b1267-8d89-4024-8b0b-568c8d800cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:24.749997 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b1267-8d89-4024-8b0b-568c8d800cd0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:24.754026 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e20b1c-d9b9-455b-8feb-b5cc1128de95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:24.754641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e20b1c-d9b9-455b-8feb-b5cc1128de95 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.754926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e20b1c-d9b9-455b-8feb-b5cc1128de95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.755205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e20b1c-d9b9-455b-8feb-b5cc1128de95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.764108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e20b1c-d9b9-455b-8feb-b5cc1128de95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.765617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e20b1c-d9b9-455b-8feb-b5cc1128de95 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.770086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:24.775224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1325/2663] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:24.797052 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94692c-a88b-4318-8382-e376744018e6 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:24.859680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94692c-a88b-4318-8382-e376744018e6 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:24.864204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:24.864858 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.865147 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.865420 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] PATH_INFO: `/v3/users/52b7819c1db848059cb5776e5b6d30b0` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.866551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52b7819c1db848059cb5776e5b6d30b0)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:24.869185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:24.914151 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52b7819c1db848059cb5776e5b6d30b0'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.915314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52b7819c1db848059cb5776e5b6d30b0'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.923042 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52b7819c1db848059cb5776e5b6d30b0'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.923640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52b7819c1db848059cb5776e5b6d30b0'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.931272 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52b7819c1db848059cb5776e5b6d30b0'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:24.949138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1339/2664] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:24 2022] DELETE /identity/v3/users/52b7819c1db848059cb5776e5b6d30b0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:24.953541 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa01bd-78d2-4dcc-8c4d-e988a36f3da3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:24.954006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa01bd-78d2-4dcc-8c4d-e988a36f3da3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:24.954098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa01bd-78d2-4dcc-8c4d-e988a36f3da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:24.973054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-eefa01bd-78d2-4dcc-8c4d-e988a36f3da3 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:24.974243 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eefa01bd-78d2-4dcc-8c4d-e988a36f3da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:25.033818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1326/2665] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:25.044144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:25.045584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:25.101662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0eb62-b6a4-409b-ae44-0a4076d99472 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:25.105865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463f468-330d-481a-82ad-ab8f52f981a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:25.106453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f468-330d-481a-82ad-ab8f52f981a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:25.106728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f468-330d-481a-82ad-ab8f52f981a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:25.106995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f468-330d-481a-82ad-ab8f52f981a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:25.115766 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463f468-330d-481a-82ad-ab8f52f981a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:25.117166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463f468-330d-481a-82ad-ab8f52f981a1 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:25.126390 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1340/2666] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:25.291151 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa01bd-78d2-4dcc-8c4d-e988a36f3da3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:25.292598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eefa01bd-78d2-4dcc-8c4d-e988a36f3da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:25.316404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa01bd-78d2-4dcc-8c4d-e988a36f3da3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:25.323984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:25.325031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:25.325562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:25.325973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] PATH_INFO: `/v3/users/efaed26081034f82b9743415dedc5b54` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:25.327396 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efaed26081034f82b9743415dedc5b54)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:25.330023 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:25.387638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efaed26081034f82b9743415dedc5b54'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.399848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efaed26081034f82b9743415dedc5b54'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.400575 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efaed26081034f82b9743415dedc5b54'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.411044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efaed26081034f82b9743415dedc5b54'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.412226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efaed26081034f82b9743415dedc5b54'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.437060 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1327/2667] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:25 2022] DELETE /identity/v3/users/efaed26081034f82b9743415dedc5b54 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:25.440114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bd7c8-56bb-48a4-8c78-ac6ea020613c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:25.440432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bd7c8-56bb-48a4-8c78-ac6ea020613c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:25.440745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bd7c8-56bb-48a4-8c78-ac6ea020613c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:25.451418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:25.454693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:25.466209 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-337bd7c8-56bb-48a4-8c78-ac6ea020613c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:25.467960 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-337bd7c8-56bb-48a4-8c78-ac6ea020613c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:25.488893 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:25.546082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1341/2668] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:25.556576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bd7c8-56bb-48a4-8c78-ac6ea020613c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:25.561886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-337bd7c8-56bb-48a4-8c78-ac6ea020613c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:25.568579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1631a1-d612-495f-8319-a44a7c7ec9f1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:25.574126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:25.575102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:25.575370 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:25.575739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] PATH_INFO: `/v3/projects/83ab212541a84c40917cf7cbeaf0790a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:25.577418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ab212541a84c40917cf7cbeaf0790a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:25.579313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:25.597237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:25.628970 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bd7c8-56bb-48a4-8c78-ac6ea020613c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:25.637009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ab212541a84c40917cf7cbeaf0790a', 'request_id': 'req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.643716 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ab212541a84c40917cf7cbeaf0790a', 'request_id': 'req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.652113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ab212541a84c40917cf7cbeaf0790a', 'request_id': 'req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.664588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] The token cache is being invalidate because project 83ab212541a84c40917cf7cbeaf0790a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:25.665910 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.666725 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7aa039e1-f467-47fb-bc47-511a3a3dfa5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:25.668345 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1328/2669] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:25 2022] DELETE /identity/v3/projects/83ab212541a84c40917cf7cbeaf0790a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:25.673179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eac9c6-30a7-4354-9aec-a1ca2cd02fd5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:25.673458 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eac9c6-30a7-4354-9aec-a1ca2cd02fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:25.673724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eac9c6-30a7-4354-9aec-a1ca2cd02fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:25.691150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31073be0-b08e-4e06-81af-fb769db7e2ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:25.691764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31073be0-b08e-4e06-81af-fb769db7e2ea service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:25.692044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31073be0-b08e-4e06-81af-fb769db7e2ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:25.692317 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31073be0-b08e-4e06-81af-fb769db7e2ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:25.694006 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a1eac9c6-30a7-4354-9aec-a1ca2cd02fd5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:25.694121 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-31073be0-b08e-4e06-81af-fb769db7e2ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:25.695180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a1eac9c6-30a7-4354-9aec-a1ca2cd02fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:25.781724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1329/2670] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:25.789203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47384f8c-386e-46bc-bae1-e625e414f4d9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:25.789564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47384f8c-386e-46bc-bae1-e625e414f4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:25.790120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47384f8c-386e-46bc-bae1-e625e414f4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:25.794805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31073be0-b08e-4e06-81af-fb769db7e2ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:25.796966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31073be0-b08e-4e06-81af-fb769db7e2ea service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:25.811962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1342/2671] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:25.812859 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-47384f8c-386e-46bc-bae1-e625e414f4d9 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:25.814097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-47384f8c-386e-46bc-bae1-e625e414f4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:25.823232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1330/2672] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:25.832474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31073be0-b08e-4e06-81af-fb769db7e2ea service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:25.843707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31073be0-b08e-4e06-81af-fb769db7e2ea service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:25.848341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ac4d2-c2ac-4eb1-bd73-77111e6ec13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:25.848981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ac4d2-c2ac-4eb1-bd73-77111e6ec13d admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:25.849257 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ac4d2-c2ac-4eb1-bd73-77111e6ec13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:25.849540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ac4d2-c2ac-4eb1-bd73-77111e6ec13d admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:25.850168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ac4d2-c2ac-4eb1-bd73-77111e6ec13d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:25.850475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ac4d2-c2ac-4eb1-bd73-77111e6ec13d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:25.852071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ac4d2-c2ac-4eb1-bd73-77111e6ec13d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:25.865884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1343/2673] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:25.870486 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47384f8c-386e-46bc-bae1-e625e414f4d9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:25.882068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47384f8c-386e-46bc-bae1-e625e414f4d9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:25.887194 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c8f28-0f87-4594-80e3-e65c44c389a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:25.887905 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c8f28-0f87-4594-80e3-e65c44c389a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:25.888289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c8f28-0f87-4594-80e3-e65c44c389a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:25.888634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c8f28-0f87-4594-80e3-e65c44c389a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:25.889405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c8f28-0f87-4594-80e3-e65c44c389a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:25.892005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c8f28-0f87-4594-80e3-e65c44c389a2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:25.922774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1331/2674] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:25 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:25.926992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ac4d2-c2ac-4eb1-bd73-77111e6ec13d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:25.938339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ac4d2-c2ac-4eb1-bd73-77111e6ec13d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:25.951280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:26.002094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd71c1-a04b-44c6-a85e-43b82d3a23fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.002712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd71c1-a04b-44c6-a85e-43b82d3a23fa admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.002987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd71c1-a04b-44c6-a85e-43b82d3a23fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.003268 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd71c1-a04b-44c6-a85e-43b82d3a23fa admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.003914 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd71c1-a04b-44c6-a85e-43b82d3a23fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.005898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd71c1-a04b-44c6-a85e-43b82d3a23fa admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:26.032021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1344/2675] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:25 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:26.036204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c8f28-0f87-4594-80e3-e65c44c389a2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:26.046936 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c8f28-0f87-4594-80e3-e65c44c389a2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:26.051805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15dc7d-0577-4647-93b6-cd27b3f020aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.052523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15dc7d-0577-4647-93b6-cd27b3f020aa admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.052924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15dc7d-0577-4647-93b6-cd27b3f020aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.053266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15dc7d-0577-4647-93b6-cd27b3f020aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.054051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15dc7d-0577-4647-93b6-cd27b3f020aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.057186 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15dc7d-0577-4647-93b6-cd27b3f020aa admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:26.067098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1332/2676] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:26.071304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd71c1-a04b-44c6-a85e-43b82d3a23fa admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:26.081934 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd71c1-a04b-44c6-a85e-43b82d3a23fa admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:26.086640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1c117-3920-4ebd-a922-e2914042ce8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.087543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1c117-3920-4ebd-a922-e2914042ce8b admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.087826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1c117-3920-4ebd-a922-e2914042ce8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.088213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1c117-3920-4ebd-a922-e2914042ce8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.088925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1c117-3920-4ebd-a922-e2914042ce8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.090520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1c117-3920-4ebd-a922-e2914042ce8b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:26.102631 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1345/2677] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:26.107309 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15dc7d-0577-4647-93b6-cd27b3f020aa admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:26.116354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1c117-3920-4ebd-a922-e2914042ce8b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:26.119070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15dc7d-0577-4647-93b6-cd27b3f020aa admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:26.119751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-58b1c117-3920-4ebd-a922-e2914042ce8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:26.130337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67287458-0cdf-435f-a628-fee7b02e04b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.131794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287458-0cdf-435f-a628-fee7b02e04b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.132168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287458-0cdf-435f-a628-fee7b02e04b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.132350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287458-0cdf-435f-a628-fee7b02e04b4 admin admin] PATH_INFO: `/v3/projects/00340318ce594f8b98b612f102031dd2/users/803b6a4777844bc7ad5e97cc3fc00e92/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.134208 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1c117-3920-4ebd-a922-e2914042ce8b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:26.142410 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.143271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.143658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.143932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] PATH_INFO: `/v3/projects/149e3477c3a24dbfa79880e728ea8737` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.145371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=149e3477c3a24dbfa79880e728ea8737)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.147220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:26.151780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67287458-0cdf-435f-a628-fee7b02e04b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00340318ce594f8b98b612f102031dd2, user_id=803b6a4777844bc7ad5e97cc3fc00e92, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.155057 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67287458-0cdf-435f-a628-fee7b02e04b4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:26.170030 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1333/2678] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:26 2022] PUT /identity/v3/projects/00340318ce594f8b98b612f102031dd2/users/803b6a4777844bc7ad5e97cc3fc00e92/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:26.180611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e7c14-9244-428d-a853-b448b08b2a6b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.181024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e7c14-9244-428d-a853-b448b08b2a6b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.181565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e7c14-9244-428d-a853-b448b08b2a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.195809 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149e3477c3a24dbfa79880e728ea8737', 'request_id': 'req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.202770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149e3477c3a24dbfa79880e728ea8737', 'request_id': 'req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.210311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149e3477c3a24dbfa79880e728ea8737', 'request_id': 'req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.211215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0e4e7c14-9244-428d-a853-b448b08b2a6b None None] MFA Rules not processed for user `803b6a4777844bc7ad5e97cc3fc00e92`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:26.212678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0e4e7c14-9244-428d-a853-b448b08b2a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:26.222704 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] The token cache is being invalidate because project 149e3477c3a24dbfa79880e728ea8737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:26.223002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.223664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.225433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1346/2679] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:26 2022] DELETE /identity/v3/projects/149e3477c3a24dbfa79880e728ea8737 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:26.229701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:26.231199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:26.243443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:26.285148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1334/2680] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:26.296352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e7c14-9244-428d-a853-b448b08b2a6b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:26.297206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1ec35-5243-48a0-b6bc-a20aec1cd3d4 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:26.297730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0e4e7c14-9244-428d-a853-b448b08b2a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:26.301959 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.302661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.303006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.303350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] PATH_INFO: `/v3/users/572e9b6612334ee1ac6bda237182c9e5` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.304477 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=572e9b6612334ee1ac6bda237182c9e5)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.306746 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:26.330187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:26.357876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e7c14-9244-428d-a853-b448b08b2a6b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:26.358917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '572e9b6612334ee1ac6bda237182c9e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.359572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '572e9b6612334ee1ac6bda237182c9e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.361466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a907c-ff4d-47e3-897a-08fffa9847a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.362068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a907c-ff4d-47e3-897a-08fffa9847a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.362346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a907c-ff4d-47e3-897a-08fffa9847a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.362609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a907c-ff4d-47e3-897a-08fffa9847a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.368480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '572e9b6612334ee1ac6bda237182c9e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.369311 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '572e9b6612334ee1ac6bda237182c9e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.371980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a907c-ff4d-47e3-897a-08fffa9847a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.373439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a907c-ff4d-47e3-897a-08fffa9847a0 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:26.376516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '572e9b6612334ee1ac6bda237182c9e5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.382116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1335/2681] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:26.398069 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1347/2682] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:26 2022] DELETE /identity/v3/users/572e9b6612334ee1ac6bda237182c9e5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:26.703011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a907c-ff4d-47e3-897a-08fffa9847a0 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:26.721712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a907c-ff4d-47e3-897a-08fffa9847a0 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:26.790556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.791095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.791378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.791653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] PATH_INFO: `/v3/projects/b0f29268a8734772939b60d008585e4d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.792773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f29268a8734772939b60d008585e4d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.794724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:26.858404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f29268a8734772939b60d008585e4d', 'request_id': 'req-0c86a154-5713-48ab-9b84-6d9ad0cd3417'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.865917 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f29268a8734772939b60d008585e4d', 'request_id': 'req-0c86a154-5713-48ab-9b84-6d9ad0cd3417'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.875303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f29268a8734772939b60d008585e4d', 'request_id': 'req-0c86a154-5713-48ab-9b84-6d9ad0cd3417'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.887982 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] The token cache is being invalidate because project b0f29268a8734772939b60d008585e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:26.888275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.888977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:26.893811 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1336/2683] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:26 2022] DELETE /identity/v3/projects/b0f29268a8734772939b60d008585e4d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:26.894407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:26.894770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:26.895901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:26.896158 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:26.902556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:26.909428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef16db6-1b99-43e8-b991-724191104628 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:26.967421 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.968054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.968333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.968602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] PATH_INFO: `/v3/users/cb878c434a0e4866841942d0ada16449` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.968733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86a154-5713-48ab-9b84-6d9ad0cd3417 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:26.969767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb878c434a0e4866841942d0ada16449)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.971682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:26.973620 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:26.974250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:26.974547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:26.974818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] PATH_INFO: `/v3/users/9d86a1724d324e2180d0f7c8ceacc71f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:26.975946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d86a1724d324e2180d0f7c8ceacc71f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:26.977960 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:27.021815 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb878c434a0e4866841942d0ada16449'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.022485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb878c434a0e4866841942d0ada16449'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.028503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d86a1724d324e2180d0f7c8ceacc71f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.030627 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb878c434a0e4866841942d0ada16449'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.031587 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb878c434a0e4866841942d0ada16449'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.036212 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d86a1724d324e2180d0f7c8ceacc71f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.037282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d86a1724d324e2180d0f7c8ceacc71f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.040407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f3c9304a-0901-43e2-9b44-324b6f2916c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb878c434a0e4866841942d0ada16449'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.045152 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d86a1724d324e2180d0f7c8ceacc71f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.046119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d86a1724d324e2180d0f7c8ceacc71f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.059544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1348/2684] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:26 2022] DELETE /identity/v3/users/cb878c434a0e4866841942d0ada16449 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:27.065313 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1337/2685] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:26 2022] DELETE /identity/v3/users/9d86a1724d324e2180d0f7c8ceacc71f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:27.070106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf17060-436b-48a4-8f26-d2c8024b1887 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:27.070526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf17060-436b-48a4-8f26-d2c8024b1887 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:27.071442 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf17060-436b-48a4-8f26-d2c8024b1887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:27.092646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9cf17060-436b-48a4-8f26-d2c8024b1887 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:27.093867 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9cf17060-436b-48a4-8f26-d2c8024b1887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:27.105992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:27.170962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1349/2686] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:27.192178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:27.193739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:27.266884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9694d-7d26-4407-ad85-b50f6b59d2bf admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:27.271545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e210b5-5e1e-42d0-9468-67cc356dfe7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:27.272187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e210b5-5e1e-42d0-9468-67cc356dfe7b service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:27.272471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e210b5-5e1e-42d0-9468-67cc356dfe7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:27.272746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e210b5-5e1e-42d0-9468-67cc356dfe7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:27.283584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e210b5-5e1e-42d0-9468-67cc356dfe7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:27.285238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e210b5-5e1e-42d0-9468-67cc356dfe7b service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:27.296513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1338/2687] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:27.435333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf17060-436b-48a4-8f26-d2c8024b1887 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:27.436836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9cf17060-436b-48a4-8f26-d2c8024b1887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:27.449441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf17060-436b-48a4-8f26-d2c8024b1887 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:27.454873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:27.455761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:27.456218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:27.456586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] PATH_INFO: `/v3/projects/f371969888fb45e3b23ae36cb9233be1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:27.467936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f371969888fb45e3b23ae36cb9233be1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:27.470021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:27.527777 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f371969888fb45e3b23ae36cb9233be1', 'request_id': 'req-5c838264-89eb-42f6-aaae-8660872511e9'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.534016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f371969888fb45e3b23ae36cb9233be1', 'request_id': 'req-5c838264-89eb-42f6-aaae-8660872511e9'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.544414 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f371969888fb45e3b23ae36cb9233be1', 'request_id': 'req-5c838264-89eb-42f6-aaae-8660872511e9'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.560919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] The token cache is being invalidate because project f371969888fb45e3b23ae36cb9233be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:27.561407 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.562133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.563672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1350/2688] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:27 2022] DELETE /identity/v3/projects/f371969888fb45e3b23ae36cb9233be1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:27.568500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653588b9-24e6-4758-b15c-07283975f73b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:27.568930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653588b9-24e6-4758-b15c-07283975f73b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:27.569118 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653588b9-24e6-4758-b15c-07283975f73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:27.589248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-653588b9-24e6-4758-b15c-07283975f73b None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:27.590473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-653588b9-24e6-4758-b15c-07283975f73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:27.609839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:27.652586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:27.654158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:27.658077 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1339/2689] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:27.668408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838264-89eb-42f6-aaae-8660872511e9 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:27.673499 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:27.674129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:27.674262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1920bf-35bd-4e25-970c-e02ffb64dd07 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:27.674468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:27.674578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1920bf-35bd-4e25-970c-e02ffb64dd07 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:27.674662 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] PATH_INFO: `/v3/projects/f893d46cab514bde9f2eaf943019872c` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:27.674838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1920bf-35bd-4e25-970c-e02ffb64dd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:27.675850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] RBAC: Authorizing `identity:delete_project(project_id=f893d46cab514bde9f2eaf943019872c)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:27.677835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:27.696626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-fd1920bf-35bd-4e25-970c-e02ffb64dd07 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:27.697881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fd1920bf-35bd-4e25-970c-e02ffb64dd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:27.706717 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1340/2690] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:27.711665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1920bf-35bd-4e25-970c-e02ffb64dd07 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:27.721041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1920bf-35bd-4e25-970c-e02ffb64dd07 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:27.726745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f893d46cab514bde9f2eaf943019872c', 'request_id': 'req-c7a32d15-5b01-453e-b4e6-065e0ee2bced'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.733445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f893d46cab514bde9f2eaf943019872c', 'request_id': 'req-c7a32d15-5b01-453e-b4e6-065e0ee2bced'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.742479 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f893d46cab514bde9f2eaf943019872c', 'request_id': 'req-c7a32d15-5b01-453e-b4e6-065e0ee2bced'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.756066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] The token cache is being invalidate because project f893d46cab514bde9f2eaf943019872c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:27.756356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.757038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.759286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1351/2691] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:27 2022] DELETE /identity/v3/projects/f893d46cab514bde9f2eaf943019872c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:27.763074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:27.764397 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:27.783005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:27.783624 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:27.783918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:27.784159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:27.784818 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:27.785130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:27.786726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:27.793391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:27.808179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1341/2692] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:27.808956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:27.810846 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:27.825763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8c00-078d-4cf8-b6b0-ff780d063141 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:27.830827 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d4f60-9b3a-4570-9ef7-6391e4478d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:27.831481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d4f60-9b3a-4570-9ef7-6391e4478d66 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:27.831991 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d4f60-9b3a-4570-9ef7-6391e4478d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:27.832346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d4f60-9b3a-4570-9ef7-6391e4478d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:27.832976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d4f60-9b3a-4570-9ef7-6391e4478d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:27.835661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d4f60-9b3a-4570-9ef7-6391e4478d66 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:27.845513 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a32d15-5b01-453e-b4e6-065e0ee2bced admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:27.851556 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:27.852392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:27.852680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:27.853036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] PATH_INFO: `/v3/users/373c51db9c3040038917be942276b1fc` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:27.854346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=373c51db9c3040038917be942276b1fc)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:27.856734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:27.870884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1342/2693] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:27 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:27.875102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d4f60-9b3a-4570-9ef7-6391e4478d66 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:27.916402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '373c51db9c3040038917be942276b1fc'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.917238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '373c51db9c3040038917be942276b1fc'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.928275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '373c51db9c3040038917be942276b1fc'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.928877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '373c51db9c3040038917be942276b1fc'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.937964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '373c51db9c3040038917be942276b1fc'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:27.970371 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1352/2694] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:27 2022] DELETE /identity/v3/users/373c51db9c3040038917be942276b1fc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:28.000129 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d4f60-9b3a-4570-9ef7-6391e4478d66 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.064133 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:28.094645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ece006-3d46-4bd3-b9e5-3458a417cfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.095391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ece006-3d46-4bd3-b9e5-3458a417cfb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.095647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ece006-3d46-4bd3-b9e5-3458a417cfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.095932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ece006-3d46-4bd3-b9e5-3458a417cfb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.097315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ece006-3d46-4bd3-b9e5-3458a417cfb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.098599 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ece006-3d46-4bd3-b9e5-3458a417cfb2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.134801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1343/2695] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:27 2022] POST /identity/v3/users => generated 287 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:28.139202 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.140726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:28.155181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd9764-5c2d-4652-855f-6f08660ca550 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.159874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2f158-a153-4b9e-bb7a-c8ae20a29e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.160519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2f158-a153-4b9e-bb7a-c8ae20a29e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.160754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2f158-a153-4b9e-bb7a-c8ae20a29e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.161070 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2f158-a153-4b9e-bb7a-c8ae20a29e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.161712 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2f158-a153-4b9e-bb7a-c8ae20a29e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.163341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2f158-a153-4b9e-bb7a-c8ae20a29e03 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.175338 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1353/2696] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:28.179485 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ece006-3d46-4bd3-b9e5-3458a417cfb2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.190053 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ece006-3d46-4bd3-b9e5-3458a417cfb2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.195165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0cf29-18e9-4165-aa23-b3d6033ef096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.195822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0cf29-18e9-4165-aa23-b3d6033ef096 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.196308 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0cf29-18e9-4165-aa23-b3d6033ef096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.196641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0cf29-18e9-4165-aa23-b3d6033ef096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.197318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0cf29-18e9-4165-aa23-b3d6033ef096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.198936 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0cf29-18e9-4165-aa23-b3d6033ef096 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.212858 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1344/2697] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:28.214184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2f158-a153-4b9e-bb7a-c8ae20a29e03 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.224785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2f158-a153-4b9e-bb7a-c8ae20a29e03 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.230947 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf1b40-5cac-490c-8e0a-2aaa433ab13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.231595 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf1b40-5cac-490c-8e0a-2aaa433ab13f admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.231870 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf1b40-5cac-490c-8e0a-2aaa433ab13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.232149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf1b40-5cac-490c-8e0a-2aaa433ab13f admin admin] PATH_INFO: `/v3/projects/809367e544b54add8b364e3568d2bcc1/users/2efc5ec2a3aa449dbc389dfd6766d030/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.247886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf1b40-5cac-490c-8e0a-2aaa433ab13f admin admin] RBAC: Authorizing `identity:create_grant(project_id=809367e544b54add8b364e3568d2bcc1, user_id=2efc5ec2a3aa449dbc389dfd6766d030, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.250745 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf1b40-5cac-490c-8e0a-2aaa433ab13f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.264664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1354/2698] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:28 2022] PUT /identity/v3/projects/809367e544b54add8b364e3568d2bcc1/users/2efc5ec2a3aa449dbc389dfd6766d030/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:28.275142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d905c1f-8005-402c-9279-ae9f328442ee None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.275418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d905c1f-8005-402c-9279-ae9f328442ee None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.275697 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d905c1f-8005-402c-9279-ae9f328442ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.295569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf1b40-5cac-490c-8e0a-2aaa433ab13f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.297588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5cbf1b40-5cac-490c-8e0a-2aaa433ab13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:28.326249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2d905c1f-8005-402c-9279-ae9f328442ee None None] MFA Rules not processed for user `2efc5ec2a3aa449dbc389dfd6766d030`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:28.327627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2d905c1f-8005-402c-9279-ae9f328442ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:28.381261 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf1b40-5cac-490c-8e0a-2aaa433ab13f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.387153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.387911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.388184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.388451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] PATH_INFO: `/v3/projects/e9f7734f7745496ab7885a77a7509612` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.389603 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f7734f7745496ab7885a77a7509612)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.391501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.401034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:28.405533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1345/2699] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:28.413603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d905c1f-8005-402c-9279-ae9f328442ee None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.416814 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2d905c1f-8005-402c-9279-ae9f328442ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:28.460794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f7734f7745496ab7885a77a7509612', 'request_id': 'req-92f47b5c-2429-4f48-be7b-5310b1f64d70'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:28.468193 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f7734f7745496ab7885a77a7509612', 'request_id': 'req-92f47b5c-2429-4f48-be7b-5310b1f64d70'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:28.477438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f7734f7745496ab7885a77a7509612', 'request_id': 'req-92f47b5c-2429-4f48-be7b-5310b1f64d70'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:28.485501 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d905c1f-8005-402c-9279-ae9f328442ee None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.491392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] The token cache is being invalidate because project e9f7734f7745496ab7885a77a7509612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:28.491682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:28.492351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-92f47b5c-2429-4f48-be7b-5310b1f64d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:28.497565 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1355/2700] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:28 2022] DELETE /identity/v3/projects/e9f7734f7745496ab7885a77a7509612 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:28.499346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33c06b-c9ae-499b-beb9-dc941948d873 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.499734 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33c06b-c9ae-499b-beb9-dc941948d873 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.500008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33c06b-c9ae-499b-beb9-dc941948d873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.528163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4c33c06b-c9ae-499b-beb9-dc941948d873 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:28.528163 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4c33c06b-c9ae-499b-beb9-dc941948d873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:28.566404 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea345e02-dc55-46c7-a183-a8005e9fa47b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.567089 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea345e02-dc55-46c7-a183-a8005e9fa47b service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.567434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea345e02-dc55-46c7-a183-a8005e9fa47b service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.567769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea345e02-dc55-46c7-a183-a8005e9fa47b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.569715 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ea345e02-dc55-46c7-a183-a8005e9fa47b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:28.587221 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:28.601633 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1356/2701] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:28.608817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7314cb5-bfd7-4216-933c-c4f78821700c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.609133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7314cb5-bfd7-4216-933c-c4f78821700c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.609392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7314cb5-bfd7-4216-933c-c4f78821700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.628594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e7314cb5-bfd7-4216-933c-c4f78821700c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:28.629853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e7314cb5-bfd7-4216-933c-c4f78821700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:28.636647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea345e02-dc55-46c7-a183-a8005e9fa47b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.638120 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea345e02-dc55-46c7-a183-a8005e9fa47b service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.638392 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1357/2702] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:28.642562 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7314cb5-bfd7-4216-933c-c4f78821700c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.647576 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1346/2703] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:28.652413 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7314cb5-bfd7-4216-933c-c4f78821700c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.657314 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b4767-74fc-4154-bfea-011167c31d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.657911 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b4767-74fc-4154-bfea-011167c31d18 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.658191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b4767-74fc-4154-bfea-011167c31d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.658461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b4767-74fc-4154-bfea-011167c31d18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.659149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b4767-74fc-4154-bfea-011167c31d18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:28.659457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b4767-74fc-4154-bfea-011167c31d18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.661244 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b4767-74fc-4154-bfea-011167c31d18 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.673155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1358/2704] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 11:32:28.677582 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea345e02-dc55-46c7-a183-a8005e9fa47b service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.688932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea345e02-dc55-46c7-a183-a8005e9fa47b service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.693417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355a4ba-ce62-4360-a1e7-fbd7e29a6321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.694019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355a4ba-ce62-4360-a1e7-fbd7e29a6321 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.694298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355a4ba-ce62-4360-a1e7-fbd7e29a6321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.694569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355a4ba-ce62-4360-a1e7-fbd7e29a6321 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.695200 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355a4ba-ce62-4360-a1e7-fbd7e29a6321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.697978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355a4ba-ce62-4360-a1e7-fbd7e29a6321 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.729512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1347/2705] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:28 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:28.730778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b4767-74fc-4154-bfea-011167c31d18 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.741154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b4767-74fc-4154-bfea-011167c31d18 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.801403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52befb97-5f1c-47d2-8515-a27be13f927d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.802041 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52befb97-5f1c-47d2-8515-a27be13f927d admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.802316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52befb97-5f1c-47d2-8515-a27be13f927d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.802588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52befb97-5f1c-47d2-8515-a27be13f927d admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.803211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52befb97-5f1c-47d2-8515-a27be13f927d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.805696 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52befb97-5f1c-47d2-8515-a27be13f927d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.827124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1359/2706] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:28 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:28.831386 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355a4ba-ce62-4360-a1e7-fbd7e29a6321 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.841978 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355a4ba-ce62-4360-a1e7-fbd7e29a6321 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.846919 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786a7ca-b089-48bf-90a5-3e76bbba22b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.847519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786a7ca-b089-48bf-90a5-3e76bbba22b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.847799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786a7ca-b089-48bf-90a5-3e76bbba22b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.848076 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786a7ca-b089-48bf-90a5-3e76bbba22b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.848712 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786a7ca-b089-48bf-90a5-3e76bbba22b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.850350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786a7ca-b089-48bf-90a5-3e76bbba22b0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.864364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1348/2707] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:28.865844 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52befb97-5f1c-47d2-8515-a27be13f927d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.876599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52befb97-5f1c-47d2-8515-a27be13f927d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.881239 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c2776-caa2-4a47-a3d8-d67172ec8365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.881837 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c2776-caa2-4a47-a3d8-d67172ec8365 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.882113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c2776-caa2-4a47-a3d8-d67172ec8365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.882384 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c2776-caa2-4a47-a3d8-d67172ec8365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.883023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c2776-caa2-4a47-a3d8-d67172ec8365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.884783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c2776-caa2-4a47-a3d8-d67172ec8365 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.886278 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:28.899506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1360/2708] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:28.903855 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786a7ca-b089-48bf-90a5-3e76bbba22b0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:28.914239 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786a7ca-b089-48bf-90a5-3e76bbba22b0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:28.919202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ddf99-6ffd-45c2-8b31-7c5860e28687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:28.919880 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ddf99-6ffd-45c2-8b31-7c5860e28687 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.920160 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ddf99-6ffd-45c2-8b31-7c5860e28687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.920488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ddf99-6ffd-45c2-8b31-7c5860e28687 admin admin] PATH_INFO: `/v3/projects/b9b953b27e064f9d8a6f8dabddfcf287/users/cc04aa1d2f904fe58c3c45619c7d9c89/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.936759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ddf99-6ffd-45c2-8b31-7c5860e28687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b953b27e064f9d8a6f8dabddfcf287, user_id=cc04aa1d2f904fe58c3c45619c7d9c89, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:28.939646 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ddf99-6ffd-45c2-8b31-7c5860e28687 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:28.954535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1349/2709] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:28 2022] PUT /identity/v3/projects/b9b953b27e064f9d8a6f8dabddfcf287/users/cc04aa1d2f904fe58c3c45619c7d9c89/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:28.965118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b6d54-013b-45c9-af2c-c0ef32972de3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:28.965664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b6d54-013b-45c9-af2c-c0ef32972de3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:28.965982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b6d54-013b-45c9-af2c-c0ef32972de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:28.996600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-f71b6d54-013b-45c9-af2c-c0ef32972de3 None None] MFA Rules not processed for user `cc04aa1d2f904fe58c3c45619c7d9c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:28.997901 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f71b6d54-013b-45c9-af2c-c0ef32972de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:29.077628 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1361/2710] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:29.082359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ddf99-6ffd-45c2-8b31-7c5860e28687 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:29.145319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ddf99-6ffd-45c2-8b31-7c5860e28687 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:29.150700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3968b43-5596-4fdd-984b-a02f3137035f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:29.151314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3968b43-5596-4fdd-984b-a02f3137035f admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:29.151590 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3968b43-5596-4fdd-984b-a02f3137035f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:29.151890 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3968b43-5596-4fdd-984b-a02f3137035f admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:29.152541 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3968b43-5596-4fdd-984b-a02f3137035f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:29.154464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3968b43-5596-4fdd-984b-a02f3137035f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:29.187226 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1350/2711] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:29 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:29.188383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b6d54-013b-45c9-af2c-c0ef32972de3 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:29.199139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b6d54-013b-45c9-af2c-c0ef32972de3 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:29.255138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c015178-8989-48c3-926b-3ef5a17d5324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:29.255748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c015178-8989-48c3-926b-3ef5a17d5324 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:29.256028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c015178-8989-48c3-926b-3ef5a17d5324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:29.256297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c015178-8989-48c3-926b-3ef5a17d5324 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:29.256951 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c015178-8989-48c3-926b-3ef5a17d5324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:29.259204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c015178-8989-48c3-926b-3ef5a17d5324 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:29.280412 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1362/2712] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:29 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:29.284464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3968b43-5596-4fdd-984b-a02f3137035f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:29.295315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3968b43-5596-4fdd-984b-a02f3137035f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:29.299907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae46c56-5209-407b-a8aa-47d3f7963b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:29.300536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae46c56-5209-407b-a8aa-47d3f7963b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:29.300849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae46c56-5209-407b-a8aa-47d3f7963b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:29.301127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae46c56-5209-407b-a8aa-47d3f7963b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:29.301767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae46c56-5209-407b-a8aa-47d3f7963b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:29.303368 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae46c56-5209-407b-a8aa-47d3f7963b0d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:29.314733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1351/2713] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:29.318864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c015178-8989-48c3-926b-3ef5a17d5324 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:29.329388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c015178-8989-48c3-926b-3ef5a17d5324 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:29.333931 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd863-3000-40d1-bce1-0c88a8e5dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:29.334604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd863-3000-40d1-bce1-0c88a8e5dec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:29.334897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd863-3000-40d1-bce1-0c88a8e5dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:29.335148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd863-3000-40d1-bce1-0c88a8e5dec0 admin admin] PATH_INFO: `/v3/projects/43fdb09a2fc348719a7d43286095f1e3/users/8d84c060692046bbae60a45fdf9a6c70/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:29.350706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bd863-3000-40d1-bce1-0c88a8e5dec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43fdb09a2fc348719a7d43286095f1e3, user_id=8d84c060692046bbae60a45fdf9a6c70, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:29.353935 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bd863-3000-40d1-bce1-0c88a8e5dec0 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:29.366419 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1363/2714] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:29 2022] PUT /identity/v3/projects/43fdb09a2fc348719a7d43286095f1e3/users/8d84c060692046bbae60a45fdf9a6c70/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:29.370908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae46c56-5209-407b-a8aa-47d3f7963b0d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:29.380875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae46c56-5209-407b-a8aa-47d3f7963b0d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:29.409560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:29.440783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6f38d-8656-4deb-880c-95d136afcf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:29.441422 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6f38d-8656-4deb-880c-95d136afcf7f admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:29.441707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6f38d-8656-4deb-880c-95d136afcf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:29.441980 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6f38d-8656-4deb-880c-95d136afcf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:29.442617 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6f38d-8656-4deb-880c-95d136afcf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:29.444235 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6f38d-8656-4deb-880c-95d136afcf7f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:29.456405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1352/2715] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:29.460617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd863-3000-40d1-bce1-0c88a8e5dec0 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:29.471139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd863-3000-40d1-bce1-0c88a8e5dec0 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:29.475818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0341e0a-cfee-4de7-9254-f50c35ef82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:29.476461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0341e0a-cfee-4de7-9254-f50c35ef82d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:29.476733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0341e0a-cfee-4de7-9254-f50c35ef82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:29.477616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0341e0a-cfee-4de7-9254-f50c35ef82d8 admin admin] PATH_INFO: `/v3/projects/43fdb09a2fc348719a7d43286095f1e3/users/8d84c060692046bbae60a45fdf9a6c70/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:29.479521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0341e0a-cfee-4de7-9254-f50c35ef82d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43fdb09a2fc348719a7d43286095f1e3, user_id=8d84c060692046bbae60a45fdf9a6c70, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:29.482195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0341e0a-cfee-4de7-9254-f50c35ef82d8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:29.492433 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1364/2716] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:29 2022] PUT /identity/v3/projects/43fdb09a2fc348719a7d43286095f1e3/users/8d84c060692046bbae60a45fdf9a6c70/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:29.504414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b7267-3693-4c39-bc80-ce6a60567fb3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:29.504706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b7267-3693-4c39-bc80-ce6a60567fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:29.505011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b7267-3693-4c39-bc80-ce6a60567fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:29.533321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-497b7267-3693-4c39-bc80-ce6a60567fb3 None None] MFA Rules not processed for user `8d84c060692046bbae60a45fdf9a6c70`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:29.534536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-497b7267-3693-4c39-bc80-ce6a60567fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:29.605656 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1353/2717] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:29.616784 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0341e0a-cfee-4de7-9254-f50c35ef82d8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:29.618804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d0341e0a-cfee-4de7-9254-f50c35ef82d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:29.662791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:29.679920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0341e0a-cfee-4de7-9254-f50c35ef82d8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:29.683869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87e236-04d4-4c92-bac8-99e9396a26b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:29.684462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87e236-04d4-4c92-bac8-99e9396a26b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:29.684736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87e236-04d4-4c92-bac8-99e9396a26b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:29.685032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87e236-04d4-4c92-bac8-99e9396a26b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:29.693755 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87e236-04d4-4c92-bac8-99e9396a26b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:29.695183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87e236-04d4-4c92-bac8-99e9396a26b8 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:29.704912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1365/2718] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:29.750961 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b7267-3693-4c39-bc80-ce6a60567fb3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:29.752410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-497b7267-3693-4c39-bc80-ce6a60567fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:29.764542 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b7267-3693-4c39-bc80-ce6a60567fb3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:29.768213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efae425-589c-48ad-91a9-50bafa1ed152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:29.768812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efae425-589c-48ad-91a9-50bafa1ed152 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:29.769719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efae425-589c-48ad-91a9-50bafa1ed152 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:29.770000 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efae425-589c-48ad-91a9-50bafa1ed152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:29.826413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efae425-589c-48ad-91a9-50bafa1ed152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:29.827860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efae425-589c-48ad-91a9-50bafa1ed152 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:29.837454 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1354/2719] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:33.214655 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87e236-04d4-4c92-bac8-99e9396a26b8 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:33.297941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87e236-04d4-4c92-bac8-99e9396a26b8 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:33.303550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:33.304469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:33.305518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:33.305898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] PATH_INFO: `/v3/users/cc04aa1d2f904fe58c3c45619c7d9c89` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:33.307436 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc04aa1d2f904fe58c3c45619c7d9c89)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:33.309503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:33.362993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc04aa1d2f904fe58c3c45619c7d9c89'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:33.364001 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc04aa1d2f904fe58c3c45619c7d9c89'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:33.372214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc04aa1d2f904fe58c3c45619c7d9c89'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:33.373093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc04aa1d2f904fe58c3c45619c7d9c89'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:33.382546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc04aa1d2f904fe58c3c45619c7d9c89'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:33.405728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1366/2720] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:33 2022] DELETE /identity/v3/users/cc04aa1d2f904fe58c3c45619c7d9c89 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:33.409951 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54321fd2-76d1-4903-b2bb-ae74f161d1cf None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:33.410426 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54321fd2-76d1-4903-b2bb-ae74f161d1cf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:33.410508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54321fd2-76d1-4903-b2bb-ae74f161d1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:33.431995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-54321fd2-76d1-4903-b2bb-ae74f161d1cf None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:33.433887 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-54321fd2-76d1-4903-b2bb-ae74f161d1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:33.496409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:33.511086 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1355/2721] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:33.524424 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:33.526080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:33.621351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedde57a-b0bf-427e-8846-386b61390418 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:33.627962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33481d-51a4-4bbc-a228-3d032636588f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:33.629041 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33481d-51a4-4bbc-a228-3d032636588f service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:33.629699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33481d-51a4-4bbc-a228-3d032636588f service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:33.630147 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33481d-51a4-4bbc-a228-3d032636588f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:33.651074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33481d-51a4-4bbc-a228-3d032636588f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:33.653449 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33481d-51a4-4bbc-a228-3d032636588f service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:33.666523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1367/2722] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:33.951238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54321fd2-76d1-4903-b2bb-ae74f161d1cf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:33.953292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-54321fd2-76d1-4903-b2bb-ae74f161d1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:33.972136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54321fd2-76d1-4903-b2bb-ae74f161d1cf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:33.977734 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:33.978583 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:33.978864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:33.979141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] PATH_INFO: `/v3/projects/b9b953b27e064f9d8a6f8dabddfcf287` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:33.980253 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9b953b27e064f9d8a6f8dabddfcf287)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:33.982188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:34.044427 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9b953b27e064f9d8a6f8dabddfcf287', 'request_id': 'req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.046464 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9b953b27e064f9d8a6f8dabddfcf287', 'request_id': 'req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.054282 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9b953b27e064f9d8a6f8dabddfcf287', 'request_id': 'req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.067481 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] The token cache is being invalidate because project b9b953b27e064f9d8a6f8dabddfcf287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:34.067772 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.068444 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.069962 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1356/2723] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:33 2022] DELETE /identity/v3/projects/b9b953b27e064f9d8a6f8dabddfcf287 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:34.074489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33481d-51a4-4bbc-a228-3d032636588f service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:34.079522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:34.084833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33481d-51a4-4bbc-a228-3d032636588f service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:34.150531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:34.151161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:34.151438 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:34.151706 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] PATH_INFO: `/v3/users/8d84c060692046bbae60a45fdf9a6c70` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:34.152865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d84c060692046bbae60a45fdf9a6c70)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:34.154974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:34.205559 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d84c060692046bbae60a45fdf9a6c70'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.206265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d84c060692046bbae60a45fdf9a6c70'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.214584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d84c060692046bbae60a45fdf9a6c70'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.215258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d84c060692046bbae60a45fdf9a6c70'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.222930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-c9fdbfe6-27da-4c1f-acb2-a33edf4e0675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d84c060692046bbae60a45fdf9a6c70'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.241976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1368/2724] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:34 2022] DELETE /identity/v3/users/8d84c060692046bbae60a45fdf9a6c70 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:34.524625 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:34.526124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:34.540892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9a11c-3e1b-4f0f-b450-6e7525266e07 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:34.598798 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:34.599424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:34.599701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:34.599969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] PATH_INFO: `/v3/projects/43fdb09a2fc348719a7d43286095f1e3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:34.601100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43fdb09a2fc348719a7d43286095f1e3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:34.603069 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:34.647354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43fdb09a2fc348719a7d43286095f1e3', 'request_id': 'req-123f0864-543b-4b5d-abf9-a3c99cabb426'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.653863 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43fdb09a2fc348719a7d43286095f1e3', 'request_id': 'req-123f0864-543b-4b5d-abf9-a3c99cabb426'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.662173 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43fdb09a2fc348719a7d43286095f1e3', 'request_id': 'req-123f0864-543b-4b5d-abf9-a3c99cabb426'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.677106 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] The token cache is being invalidate because project 43fdb09a2fc348719a7d43286095f1e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:34.677414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.678312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-123f0864-543b-4b5d-abf9-a3c99cabb426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:34.680685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1357/2725] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:34 2022] DELETE /identity/v3/projects/43fdb09a2fc348719a7d43286095f1e3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:34.686357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e944a7c-dab7-47fb-a86f-c31ccb8535a7 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:34.686639 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e944a7c-dab7-47fb-a86f-c31ccb8535a7 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:34.686918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e944a7c-dab7-47fb-a86f-c31ccb8535a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:34.707882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-5e944a7c-dab7-47fb-a86f-c31ccb8535a7 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:34.708979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-5e944a7c-dab7-47fb-a86f-c31ccb8535a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:34.783172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1369/2726] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:34.791137 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f584001-e710-4c2d-a7df-945e87b720fa None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:34.791560 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f584001-e710-4c2d-a7df-945e87b720fa None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:34.791844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f584001-e710-4c2d-a7df-945e87b720fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:34.807472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:34.813353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7f584001-e710-4c2d-a7df-945e87b720fa None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:34.815801 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7f584001-e710-4c2d-a7df-945e87b720fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:34.826906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1358/2727] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:34.831191 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e944a7c-dab7-47fb-a86f-c31ccb8535a7 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:34.841146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e944a7c-dab7-47fb-a86f-c31ccb8535a7 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:34.845725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c2dbc-3577-41ed-b9ef-9bf28f8bf18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:34.846337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c2dbc-3577-41ed-b9ef-9bf28f8bf18a admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:34.846616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c2dbc-3577-41ed-b9ef-9bf28f8bf18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:34.846887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c2dbc-3577-41ed-b9ef-9bf28f8bf18a admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:34.847530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c2dbc-3577-41ed-b9ef-9bf28f8bf18a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:34.847833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c2dbc-3577-41ed-b9ef-9bf28f8bf18a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:34.849462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c2dbc-3577-41ed-b9ef-9bf28f8bf18a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:34.861506 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1370/2728] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:34.866184 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f584001-e710-4c2d-a7df-945e87b720fa None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:34.875969 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f584001-e710-4c2d-a7df-945e87b720fa None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:34.880613 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c790a8d-ced3-4047-852f-70b14466560f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:34.881286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c790a8d-ced3-4047-852f-70b14466560f admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:34.881567 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c790a8d-ced3-4047-852f-70b14466560f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:34.881839 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c790a8d-ced3-4047-852f-70b14466560f admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:34.882465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c790a8d-ced3-4047-852f-70b14466560f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:34.885009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c790a8d-ced3-4047-852f-70b14466560f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:34.918853 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1359/2729] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:34 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:34.919928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c2dbc-3577-41ed-b9ef-9bf28f8bf18a admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:34.932018 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c2dbc-3577-41ed-b9ef-9bf28f8bf18a admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:34.938154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:35.012064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff153ce-9579-4152-a4ad-d002c71c1dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.013018 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff153ce-9579-4152-a4ad-d002c71c1dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.013347 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff153ce-9579-4152-a4ad-d002c71c1dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.013626 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff153ce-9579-4152-a4ad-d002c71c1dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.014324 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff153ce-9579-4152-a4ad-d002c71c1dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.017468 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff153ce-9579-4152-a4ad-d002c71c1dc7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.052399 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1371/2730] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:34 2022] POST /identity/v3/users => generated 295 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:35.057764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c790a8d-ced3-4047-852f-70b14466560f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:35.077341 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c790a8d-ced3-4047-852f-70b14466560f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:35.081364 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e419b67-d417-460b-a924-528ef4f45647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.082059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e419b67-d417-460b-a924-528ef4f45647 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.082705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e419b67-d417-460b-a924-528ef4f45647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.082979 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e419b67-d417-460b-a924-528ef4f45647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.083688 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e419b67-d417-460b-a924-528ef4f45647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.085461 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e419b67-d417-460b-a924-528ef4f45647 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.106499 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1360/2731] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:35.110707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff153ce-9579-4152-a4ad-d002c71c1dc7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:35.122337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff153ce-9579-4152-a4ad-d002c71c1dc7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:35.128450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa54105-b9bc-489c-9ffe-fe026e9400e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.129172 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54105-b9bc-489c-9ffe-fe026e9400e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.129482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54105-b9bc-489c-9ffe-fe026e9400e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.129765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54105-b9bc-489c-9ffe-fe026e9400e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.130405 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa54105-b9bc-489c-9ffe-fe026e9400e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.132205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa54105-b9bc-489c-9ffe-fe026e9400e5 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.144738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1372/2732] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:35.149342 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e419b67-d417-460b-a924-528ef4f45647 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:35.159749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e419b67-d417-460b-a924-528ef4f45647 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:35.165733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4d58a-e749-4f24-94ed-6d8601595717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.166393 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4d58a-e749-4f24-94ed-6d8601595717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.166809 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4d58a-e749-4f24-94ed-6d8601595717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.167153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4d58a-e749-4f24-94ed-6d8601595717 admin admin] PATH_INFO: `/v3/projects/2bdbb8ea223d420fb68a9d3b91e6e34b/users/853ddb7a06654bf6ae819de4c0f20003/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.184479 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4d58a-e749-4f24-94ed-6d8601595717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bdbb8ea223d420fb68a9d3b91e6e34b, user_id=853ddb7a06654bf6ae819de4c0f20003, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.187208 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4d58a-e749-4f24-94ed-6d8601595717 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.200110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1361/2733] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:35 2022] PUT /identity/v3/projects/2bdbb8ea223d420fb68a9d3b91e6e34b/users/853ddb7a06654bf6ae819de4c0f20003/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:35.211882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e14af4-f7f3-4fa2-b0d2-2afc630d1f5c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.212165 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e14af4-f7f3-4fa2-b0d2-2afc630d1f5c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.212443 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e14af4-f7f3-4fa2-b0d2-2afc630d1f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.241476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-72e14af4-f7f3-4fa2-b0d2-2afc630d1f5c None None] MFA Rules not processed for user `853ddb7a06654bf6ae819de4c0f20003`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:35.243259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-72e14af4-f7f3-4fa2-b0d2-2afc630d1f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:35.312432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1373/2734] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:35.317213 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4d58a-e749-4f24-94ed-6d8601595717 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:35.392360 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4d58a-e749-4f24-94ed-6d8601595717 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:35.398250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf16aa0-8eb1-40bf-be05-8c6afa920b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.399050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf16aa0-8eb1-40bf-be05-8c6afa920b1a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.399442 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf16aa0-8eb1-40bf-be05-8c6afa920b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.399789 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf16aa0-8eb1-40bf-be05-8c6afa920b1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.400647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf16aa0-8eb1-40bf-be05-8c6afa920b1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.403317 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf16aa0-8eb1-40bf-be05-8c6afa920b1a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.439014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1362/2735] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:35 2022] POST /identity/v3/projects => generated 393 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:35.443120 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e14af4-f7f3-4fa2-b0d2-2afc630d1f5c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:35.457096 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e14af4-f7f3-4fa2-b0d2-2afc630d1f5c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:35.523315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685eca-7140-495c-a92c-8b2bea10c554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.524038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685eca-7140-495c-a92c-8b2bea10c554 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.524437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685eca-7140-495c-a92c-8b2bea10c554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.525053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685eca-7140-495c-a92c-8b2bea10c554 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.525954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4685eca-7140-495c-a92c-8b2bea10c554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.527895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4685eca-7140-495c-a92c-8b2bea10c554 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.560771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1374/2736] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:35 2022] POST /identity/v3/users => generated 303 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:35.561995 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf16aa0-8eb1-40bf-be05-8c6afa920b1a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:35.573755 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf16aa0-8eb1-40bf-be05-8c6afa920b1a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:35.578407 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77e093-6e04-4c18-aebd-2731c904b20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.579155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77e093-6e04-4c18-aebd-2731c904b20d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.579824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77e093-6e04-4c18-aebd-2731c904b20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.579824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77e093-6e04-4c18-aebd-2731c904b20d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.580328 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77e093-6e04-4c18-aebd-2731c904b20d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.581958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77e093-6e04-4c18-aebd-2731c904b20d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.594451 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1363/2737] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:35.597768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685eca-7140-495c-a92c-8b2bea10c554 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:35.608679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685eca-7140-495c-a92c-8b2bea10c554 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:35.613904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f533b-33bb-44d1-9724-72ced5593af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.614570 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f533b-33bb-44d1-9724-72ced5593af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.614853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f533b-33bb-44d1-9724-72ced5593af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.615132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f533b-33bb-44d1-9724-72ced5593af5 admin admin] PATH_INFO: `/v3/projects/8db8856a3d0244d591e8d266ed4c38d8/users/ef3ca22a94d1489ba5515f77a195cda6/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.618237 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:35.639864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f533b-33bb-44d1-9724-72ced5593af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db8856a3d0244d591e8d266ed4c38d8, user_id=ef3ca22a94d1489ba5515f77a195cda6, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.642896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f533b-33bb-44d1-9724-72ced5593af5 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.663012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1375/2738] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:35 2022] PUT /identity/v3/projects/8db8856a3d0244d591e8d266ed4c38d8/users/ef3ca22a94d1489ba5515f77a195cda6/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:35.669550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77e093-6e04-4c18-aebd-2731c904b20d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:35.688643 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77e093-6e04-4c18-aebd-2731c904b20d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:35.793039 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c9247-64db-40df-8f16-4878c4e6a2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.794012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c9247-64db-40df-8f16-4878c4e6a2dc admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.794469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c9247-64db-40df-8f16-4878c4e6a2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.794928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c9247-64db-40df-8f16-4878c4e6a2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.796065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c9247-64db-40df-8f16-4878c4e6a2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.799185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c9247-64db-40df-8f16-4878c4e6a2dc admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.822742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1364/2739] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:35.824836 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f533b-33bb-44d1-9724-72ced5593af5 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:35.836542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f533b-33bb-44d1-9724-72ced5593af5 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:35.841145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a042ed1-7507-44bb-a8c2-49dc00fb30df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:35.841818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a042ed1-7507-44bb-a8c2-49dc00fb30df admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.842100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a042ed1-7507-44bb-a8c2-49dc00fb30df admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.842376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a042ed1-7507-44bb-a8c2-49dc00fb30df admin admin] PATH_INFO: `/v3/projects/8db8856a3d0244d591e8d266ed4c38d8/users/ef3ca22a94d1489ba5515f77a195cda6/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.844476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a042ed1-7507-44bb-a8c2-49dc00fb30df admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db8856a3d0244d591e8d266ed4c38d8, user_id=ef3ca22a94d1489ba5515f77a195cda6, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:35.847264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a042ed1-7507-44bb-a8c2-49dc00fb30df admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:35.865040 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1376/2740] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:35 2022] PUT /identity/v3/projects/8db8856a3d0244d591e8d266ed4c38d8/users/ef3ca22a94d1489ba5515f77a195cda6/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:35.870745 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c8eba-242e-4ad7-be76-a5eba41d0dbf None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:35.871113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c8eba-242e-4ad7-be76-a5eba41d0dbf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:35.871473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c8eba-242e-4ad7-be76-a5eba41d0dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:35.905463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:35.910255 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-849c8eba-242e-4ad7-be76-a5eba41d0dbf None None] MFA Rules not processed for user `ef3ca22a94d1489ba5515f77a195cda6`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:35.912662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-849c8eba-242e-4ad7-be76-a5eba41d0dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:35.995513 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1365/2741] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:35 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:36.006281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a042ed1-7507-44bb-a8c2-49dc00fb30df admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:36.008525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4a042ed1-7507-44bb-a8c2-49dc00fb30df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:36.067158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a042ed1-7507-44bb-a8c2-49dc00fb30df admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:36.071422 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acf22f-c473-413e-a4a3-2c57f9b1367f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:36.072028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf22f-c473-413e-a4a3-2c57f9b1367f service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:36.072305 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf22f-c473-413e-a4a3-2c57f9b1367f service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:36.072573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf22f-c473-413e-a4a3-2c57f9b1367f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:36.081873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acf22f-c473-413e-a4a3-2c57f9b1367f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:36.083461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acf22f-c473-413e-a4a3-2c57f9b1367f service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:36.096111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1377/2742] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:36 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:36.156095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c8eba-242e-4ad7-be76-a5eba41d0dbf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:36.167958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c8eba-242e-4ad7-be76-a5eba41d0dbf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:36.174049 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acf22f-c473-413e-a4a3-2c57f9b1367f service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:36.174816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f56d0-720c-4c5c-a971-31e8580a2eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:36.175465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f56d0-720c-4c5c-a971-31e8580a2eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:36.175692 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f56d0-720c-4c5c-a971-31e8580a2eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:36.175967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-53acf22f-c473-413e-a4a3-2c57f9b1367f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:36.176073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f56d0-720c-4c5c-a971-31e8580a2eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:36.241606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f56d0-720c-4c5c-a971-31e8580a2eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:36.243679 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f56d0-720c-4c5c-a971-31e8580a2eb9 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:36.253792 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1366/2743] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:36.254496 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acf22f-c473-413e-a4a3-2c57f9b1367f service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:36.259774 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:36.260360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:36.260636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:36.260941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] PATH_INFO: `/v3/users/803b6a4777844bc7ad5e97cc3fc00e92` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:36.262124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=803b6a4777844bc7ad5e97cc3fc00e92)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:36.264148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:36.319439 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '803b6a4777844bc7ad5e97cc3fc00e92'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.320111 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '803b6a4777844bc7ad5e97cc3fc00e92'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.329200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '803b6a4777844bc7ad5e97cc3fc00e92'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.329881 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '803b6a4777844bc7ad5e97cc3fc00e92'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.338878 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '803b6a4777844bc7ad5e97cc3fc00e92'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.347116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:36.365490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1378/2744] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:36 2022] DELETE /identity/v3/users/803b6a4777844bc7ad5e97cc3fc00e92 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:36.365585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b77c-9374-4144-a156-a7b1cfc0f6e0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:36.365585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b77c-9374-4144-a156-a7b1cfc0f6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:36.365585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b77c-9374-4144-a156-a7b1cfc0f6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:36.385952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1674b77c-9374-4144-a156-a7b1cfc0f6e0 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:36.387174 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1674b77c-9374-4144-a156-a7b1cfc0f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:36.456078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1367/2745] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:36.465723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:36.468616 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:36.565235 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e3e2c-2937-4ad9-ab6a-c234a4bcd59e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:36.569273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477f55e-0c43-4b5e-b97d-8e11f531a238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:36.570016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477f55e-0c43-4b5e-b97d-8e11f531a238 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:36.571003 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477f55e-0c43-4b5e-b97d-8e11f531a238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:36.571356 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477f55e-0c43-4b5e-b97d-8e11f531a238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:36.581409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477f55e-0c43-4b5e-b97d-8e11f531a238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:36.583164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477f55e-0c43-4b5e-b97d-8e11f531a238 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:36.598818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1379/2746] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:36.886110 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b77c-9374-4144-a156-a7b1cfc0f6e0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:36.888387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1674b77c-9374-4144-a156-a7b1cfc0f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:36.902821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b77c-9374-4144-a156-a7b1cfc0f6e0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:36.908218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:36.908956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:36.909319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:36.909654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] PATH_INFO: `/v3/projects/00340318ce594f8b98b612f102031dd2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:36.910968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] RBAC: Authorizing `identity:delete_project(project_id=00340318ce594f8b98b612f102031dd2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:36.913111 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:36.953059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:36.967223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00340318ce594f8b98b612f102031dd2', 'request_id': 'req-557aeeb2-65d0-4949-848e-7c5b5b78827c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.973490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00340318ce594f8b98b612f102031dd2', 'request_id': 'req-557aeeb2-65d0-4949-848e-7c5b5b78827c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.983230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00340318ce594f8b98b612f102031dd2', 'request_id': 'req-557aeeb2-65d0-4949-848e-7c5b5b78827c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.995230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] The token cache is being invalidate because project 00340318ce594f8b98b612f102031dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:36.995524 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.996168 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-557aeeb2-65d0-4949-848e-7c5b5b78827c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:36.997695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1368/2747] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:36 2022] DELETE /identity/v3/projects/00340318ce594f8b98b612f102031dd2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:37.002659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594183f-7224-4146-917b-5968567db8bf None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.003167 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594183f-7224-4146-917b-5968567db8bf None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.003473 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594183f-7224-4146-917b-5968567db8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.024034 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-4594183f-7224-4146-917b-5968567db8bf None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:37.025291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-4594183f-7224-4146-917b-5968567db8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:37.087797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1380/2748] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:37.095335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e749f1-7ddc-4fdd-8e28-863bcc5aef72 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.095624 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e749f1-7ddc-4fdd-8e28-863bcc5aef72 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.095900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e749f1-7ddc-4fdd-8e28-863bcc5aef72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.116726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-95e749f1-7ddc-4fdd-8e28-863bcc5aef72 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:37.118925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-95e749f1-7ddc-4fdd-8e28-863bcc5aef72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:37.131500 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1369/2749] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:37.132680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594183f-7224-4146-917b-5968567db8bf None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.143074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594183f-7224-4146-917b-5968567db8bf None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.149660 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2190e1c-0e09-4223-9bf2-6b1a876e2d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.150275 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2190e1c-0e09-4223-9bf2-6b1a876e2d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.150555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2190e1c-0e09-4223-9bf2-6b1a876e2d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.150827 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2190e1c-0e09-4223-9bf2-6b1a876e2d34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.151605 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2190e1c-0e09-4223-9bf2-6b1a876e2d34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:37.151917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2190e1c-0e09-4223-9bf2-6b1a876e2d34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.153590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2190e1c-0e09-4223-9bf2-6b1a876e2d34 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.166589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1381/2750] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:37.170756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e749f1-7ddc-4fdd-8e28-863bcc5aef72 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.181720 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e749f1-7ddc-4fdd-8e28-863bcc5aef72 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.187634 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba68620-ff18-45fc-b2da-e13b8e92babf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.188296 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba68620-ff18-45fc-b2da-e13b8e92babf admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.188756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba68620-ff18-45fc-b2da-e13b8e92babf admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.189155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba68620-ff18-45fc-b2da-e13b8e92babf admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.189893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba68620-ff18-45fc-b2da-e13b8e92babf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.191902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba68620-ff18-45fc-b2da-e13b8e92babf admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.224157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1370/2751] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:37 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:37.225621 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2190e1c-0e09-4223-9bf2-6b1a876e2d34 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.236112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2190e1c-0e09-4223-9bf2-6b1a876e2d34 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.308005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0c5c9-fc91-45be-ac44-4a5ddaee7a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.308724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0c5c9-fc91-45be-ac44-4a5ddaee7a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.308953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0c5c9-fc91-45be-ac44-4a5ddaee7a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.309233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0c5c9-fc91-45be-ac44-4a5ddaee7a06 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.309899 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0c5c9-fc91-45be-ac44-4a5ddaee7a06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.311902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0c5c9-fc91-45be-ac44-4a5ddaee7a06 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.341493 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1382/2752] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:37 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:37.343273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba68620-ff18-45fc-b2da-e13b8e92babf admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.354629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba68620-ff18-45fc-b2da-e13b8e92babf admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.360009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5e00f-90d6-4191-9e0b-a90307947e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.360718 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5e00f-90d6-4191-9e0b-a90307947e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.361100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5e00f-90d6-4191-9e0b-a90307947e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.361445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5e00f-90d6-4191-9e0b-a90307947e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.362172 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5e00f-90d6-4191-9e0b-a90307947e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.363878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5e00f-90d6-4191-9e0b-a90307947e97 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.375768 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1371/2753] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:37.379811 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0c5c9-fc91-45be-ac44-4a5ddaee7a06 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.391503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0c5c9-fc91-45be-ac44-4a5ddaee7a06 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.396267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762e737-df50-4cd5-8b2d-a9196858eb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.396923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762e737-df50-4cd5-8b2d-a9196858eb2a admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.397187 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762e737-df50-4cd5-8b2d-a9196858eb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.397460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762e737-df50-4cd5-8b2d-a9196858eb2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.398116 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762e737-df50-4cd5-8b2d-a9196858eb2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.400584 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762e737-df50-4cd5-8b2d-a9196858eb2a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.406483 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:37.414162 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1383/2754] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:37.422148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5e00f-90d6-4191-9e0b-a90307947e97 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.433293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5e00f-90d6-4191-9e0b-a90307947e97 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.438249 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99fb4b-d591-46eb-9390-f2e2441e1465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.438975 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99fb4b-d591-46eb-9390-f2e2441e1465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.439336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99fb4b-d591-46eb-9390-f2e2441e1465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.446700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99fb4b-d591-46eb-9390-f2e2441e1465 admin admin] PATH_INFO: `/v3/projects/11d5687cea724eba885f31c6693de8fd/users/b0cea33b99c3430ba815759162dd1ac7/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.456977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99fb4b-d591-46eb-9390-f2e2441e1465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d5687cea724eba885f31c6693de8fd, user_id=b0cea33b99c3430ba815759162dd1ac7, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.459649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99fb4b-d591-46eb-9390-f2e2441e1465 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.479337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1372/2755] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:37 2022] PUT /identity/v3/projects/11d5687cea724eba885f31c6693de8fd/users/b0cea33b99c3430ba815759162dd1ac7/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:37.490596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021997-16c4-43d4-88dd-0c202c554108 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.490596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021997-16c4-43d4-88dd-0c202c554108 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.490596 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021997-16c4-43d4-88dd-0c202c554108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.518189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e8021997-16c4-43d4-88dd-0c202c554108 None None] MFA Rules not processed for user `b0cea33b99c3430ba815759162dd1ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:37.519402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e8021997-16c4-43d4-88dd-0c202c554108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:37.596287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1384/2756] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:37.600766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99fb4b-d591-46eb-9390-f2e2441e1465 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.671589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99fb4b-d591-46eb-9390-f2e2441e1465 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.676305 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01142e3-3474-4fd2-b489-82d006653676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.676941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01142e3-3474-4fd2-b489-82d006653676 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.677224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01142e3-3474-4fd2-b489-82d006653676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.677482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01142e3-3474-4fd2-b489-82d006653676 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.678119 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01142e3-3474-4fd2-b489-82d006653676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.680452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01142e3-3474-4fd2-b489-82d006653676 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.715375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1373/2757] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:37 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:37.719050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021997-16c4-43d4-88dd-0c202c554108 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.731795 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021997-16c4-43d4-88dd-0c202c554108 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.808026 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6327-91e4-4e6d-9fe4-02d7b768f010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.808723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6327-91e4-4e6d-9fe4-02d7b768f010 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.809181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6327-91e4-4e6d-9fe4-02d7b768f010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.809530 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6327-91e4-4e6d-9fe4-02d7b768f010 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.810241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f6327-91e4-4e6d-9fe4-02d7b768f010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.812335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f6327-91e4-4e6d-9fe4-02d7b768f010 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.839869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1385/2758] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:37 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:37.844109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01142e3-3474-4fd2-b489-82d006653676 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.857993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01142e3-3474-4fd2-b489-82d006653676 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.864005 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d729b3-a78d-4a4d-b3fc-88441dbad158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.864686 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d729b3-a78d-4a4d-b3fc-88441dbad158 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.865008 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d729b3-a78d-4a4d-b3fc-88441dbad158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.865275 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d729b3-a78d-4a4d-b3fc-88441dbad158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.865915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d729b3-a78d-4a4d-b3fc-88441dbad158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.869412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d729b3-a78d-4a4d-b3fc-88441dbad158 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.884923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1374/2759] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:37.889015 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6327-91e4-4e6d-9fe4-02d7b768f010 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.899408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6327-91e4-4e6d-9fe4-02d7b768f010 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:37.904763 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041525ff-0f90-41cb-88ee-c713b1297902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:37.905466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041525ff-0f90-41cb-88ee-c713b1297902 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:37.905748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041525ff-0f90-41cb-88ee-c713b1297902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:37.906024 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041525ff-0f90-41cb-88ee-c713b1297902 admin admin] PATH_INFO: `/v3/projects/7263fe03e82747ff9ba799f88a0debfd/users/8f5b92ae97c54be7980f4298b1f0ae6c/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:37.922148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041525ff-0f90-41cb-88ee-c713b1297902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7263fe03e82747ff9ba799f88a0debfd, user_id=8f5b92ae97c54be7980f4298b1f0ae6c, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:37.925629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041525ff-0f90-41cb-88ee-c713b1297902 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:37.940394 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1386/2760] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:37 2022] PUT /identity/v3/projects/7263fe03e82747ff9ba799f88a0debfd/users/8f5b92ae97c54be7980f4298b1f0ae6c/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:37.944346 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d729b3-a78d-4a4d-b3fc-88441dbad158 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:37.958640 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d729b3-a78d-4a4d-b3fc-88441dbad158 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:38.001179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:38.031946 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643d2b1-e4ca-40c6-8eb8-aaa582418a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:38.033579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643d2b1-e4ca-40c6-8eb8-aaa582418a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:38.033874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643d2b1-e4ca-40c6-8eb8-aaa582418a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:38.034914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643d2b1-e4ca-40c6-8eb8-aaa582418a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:38.035570 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643d2b1-e4ca-40c6-8eb8-aaa582418a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:38.038016 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643d2b1-e4ca-40c6-8eb8-aaa582418a2f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:38.055465 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1375/2761] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:38.059737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041525ff-0f90-41cb-88ee-c713b1297902 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:38.070955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041525ff-0f90-41cb-88ee-c713b1297902 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:38.076431 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddc1b9-2d3e-4cc1-b182-c8ba6ea3fbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:38.077132 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddc1b9-2d3e-4cc1-b182-c8ba6ea3fbb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:38.077411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddc1b9-2d3e-4cc1-b182-c8ba6ea3fbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:38.077684 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddc1b9-2d3e-4cc1-b182-c8ba6ea3fbb7 admin admin] PATH_INFO: `/v3/projects/7263fe03e82747ff9ba799f88a0debfd/users/8f5b92ae97c54be7980f4298b1f0ae6c/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:38.079873 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddc1b9-2d3e-4cc1-b182-c8ba6ea3fbb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7263fe03e82747ff9ba799f88a0debfd, user_id=8f5b92ae97c54be7980f4298b1f0ae6c, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:38.083228 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddc1b9-2d3e-4cc1-b182-c8ba6ea3fbb7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:38.108908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24d409-61dc-45a2-bc1d-c23ecc336907 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:38.109199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24d409-61dc-45a2-bc1d-c23ecc336907 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:38.109475 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24d409-61dc-45a2-bc1d-c23ecc336907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:38.112154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1387/2762] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:38 2022] PUT /identity/v3/projects/7263fe03e82747ff9ba799f88a0debfd/users/8f5b92ae97c54be7980f4298b1f0ae6c/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:38.140462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3d24d409-61dc-45a2-bc1d-c23ecc336907 None None] MFA Rules not processed for user `8f5b92ae97c54be7980f4298b1f0ae6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:38.141894 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3d24d409-61dc-45a2-bc1d-c23ecc336907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:38.245893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1376/2763] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:38.298890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddc1b9-2d3e-4cc1-b182-c8ba6ea3fbb7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:38.300636 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-49ddc1b9-2d3e-4cc1-b182-c8ba6ea3fbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:38.366334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:38.392454 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddc1b9-2d3e-4cc1-b182-c8ba6ea3fbb7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:38.400136 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803a65d-3ebb-4c81-bff7-cc637df70ab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:38.400917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803a65d-3ebb-4c81-bff7-cc637df70ab3 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:38.401205 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803a65d-3ebb-4c81-bff7-cc637df70ab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:38.401484 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803a65d-3ebb-4c81-bff7-cc637df70ab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:38.481882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803a65d-3ebb-4c81-bff7-cc637df70ab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:38.485602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803a65d-3ebb-4c81-bff7-cc637df70ab3 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:38.497206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1388/2764] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:40.214925 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24d409-61dc-45a2-bc1d-c23ecc336907 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:40.216473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3d24d409-61dc-45a2-bc1d-c23ecc336907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:40.280519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24d409-61dc-45a2-bc1d-c23ecc336907 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:40.285385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:40.286037 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:40.286303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:40.286569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] PATH_INFO: `/v3/users/853ddb7a06654bf6ae819de4c0f20003` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:40.287707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=853ddb7a06654bf6ae819de4c0f20003)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:40.290446 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:40.337116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853ddb7a06654bf6ae819de4c0f20003'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.345324 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853ddb7a06654bf6ae819de4c0f20003'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.345945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853ddb7a06654bf6ae819de4c0f20003'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.347230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:40.357055 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853ddb7a06654bf6ae819de4c0f20003'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.357729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853ddb7a06654bf6ae819de4c0f20003'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.376237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1377/2765] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:40 2022] DELETE /identity/v3/users/853ddb7a06654bf6ae819de4c0f20003 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:40.380771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067f58b-a2dd-4f4e-813a-e3cd25a4c639 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:40.381247 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067f58b-a2dd-4f4e-813a-e3cd25a4c639 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:40.381339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067f58b-a2dd-4f4e-813a-e3cd25a4c639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:40.402552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-2067f58b-a2dd-4f4e-813a-e3cd25a4c639 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:40.403754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2067f58b-a2dd-4f4e-813a-e3cd25a4c639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:40.465428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1389/2766] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:40.475321 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:40.476899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:40.539056 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa20f8-e058-4973-917a-5444b1135a22 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:40.543041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74a20-502b-4fd5-b30f-e5ecc7beb78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:40.543603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74a20-502b-4fd5-b30f-e5ecc7beb78f service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:40.543784 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74a20-502b-4fd5-b30f-e5ecc7beb78f service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:40.544062 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74a20-502b-4fd5-b30f-e5ecc7beb78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:40.553669 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d74a20-502b-4fd5-b30f-e5ecc7beb78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:40.555130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d74a20-502b-4fd5-b30f-e5ecc7beb78f service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:40.565032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1378/2767] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:40.854159 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067f58b-a2dd-4f4e-813a-e3cd25a4c639 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:40.855716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2067f58b-a2dd-4f4e-813a-e3cd25a4c639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:40.870949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067f58b-a2dd-4f4e-813a-e3cd25a4c639 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:40.876279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:40.877161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:40.877301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:40.877516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] PATH_INFO: `/v3/projects/2bdbb8ea223d420fb68a9d3b91e6e34b` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:40.879553 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bdbb8ea223d420fb68a9d3b91e6e34b)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:40.881612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:40.934030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bdbb8ea223d420fb68a9d3b91e6e34b', 'request_id': 'req-0255809d-f20e-4877-91f5-17d70f4bfff7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.939834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bdbb8ea223d420fb68a9d3b91e6e34b', 'request_id': 'req-0255809d-f20e-4877-91f5-17d70f4bfff7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.947689 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bdbb8ea223d420fb68a9d3b91e6e34b', 'request_id': 'req-0255809d-f20e-4877-91f5-17d70f4bfff7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.963915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] The token cache is being invalidate because project 2bdbb8ea223d420fb68a9d3b91e6e34b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:40.964212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.964905 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:40.966654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1390/2768] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:40 2022] DELETE /identity/v3/projects/2bdbb8ea223d420fb68a9d3b91e6e34b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:40.970358 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74a20-502b-4fd5-b30f-e5ecc7beb78f service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:40.980455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74a20-502b-4fd5-b30f-e5ecc7beb78f service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:41.052268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:41.052940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:41.053220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:41.053596 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] PATH_INFO: `/v3/users/ef3ca22a94d1489ba5515f77a195cda6` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:41.054826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef3ca22a94d1489ba5515f77a195cda6)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:41.057063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:41.107390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef3ca22a94d1489ba5515f77a195cda6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.114815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef3ca22a94d1489ba5515f77a195cda6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.115431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef3ca22a94d1489ba5515f77a195cda6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.122632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef3ca22a94d1489ba5515f77a195cda6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.123273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2c2730c5-6282-40a6-9c20-1d80a441a2b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef3ca22a94d1489ba5515f77a195cda6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.135392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:41.144418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1379/2769] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:40 2022] DELETE /identity/v3/users/ef3ca22a94d1489ba5515f77a195cda6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:41.236606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:41.238091 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:41.253375 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255809d-f20e-4877-91f5-17d70f4bfff7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:41.274327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:41.335699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:41.336642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:41.336961 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:41.337238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] PATH_INFO: `/v3/users/b0cea33b99c3430ba815759162dd1ac7` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:41.338447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0cea33b99c3430ba815759162dd1ac7)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:41.340426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:41.399476 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0cea33b99c3430ba815759162dd1ac7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.400206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0cea33b99c3430ba815759162dd1ac7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.411475 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0cea33b99c3430ba815759162dd1ac7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.413640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0cea33b99c3430ba815759162dd1ac7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.425370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0cea33b99c3430ba815759162dd1ac7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.453129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1391/2770] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:41 2022] DELETE /identity/v3/users/b0cea33b99c3430ba815759162dd1ac7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:41.459742 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:41.460623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4302789-0038-4ece-89fa-9594d12dd691 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:41.461237 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4302789-0038-4ece-89fa-9594d12dd691 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:41.461603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4302789-0038-4ece-89fa-9594d12dd691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:41.462258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:41.504130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d4302789-0038-4ece-89fa-9594d12dd691 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:41.505491 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d4302789-0038-4ece-89fa-9594d12dd691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:41.548351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987eb9e-336c-4c6b-bd73-a0b6be6e6cf6 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:41.554298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:41.556649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:41.557033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:41.557379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] PATH_INFO: `/v3/projects/8db8856a3d0244d591e8d266ed4c38d8` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:41.559080 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db8856a3d0244d591e8d266ed4c38d8)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:41.561265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:41.579908 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1380/2771] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:41.591937 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4302789-0038-4ece-89fa-9594d12dd691 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:41.593476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d4302789-0038-4ece-89fa-9594d12dd691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:41.621682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:41.636791 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db8856a3d0244d591e8d266ed4c38d8', 'request_id': 'req-7ed2e20f-8a18-46c9-aabd-608e182220c4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.649722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db8856a3d0244d591e8d266ed4c38d8', 'request_id': 'req-7ed2e20f-8a18-46c9-aabd-608e182220c4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.668938 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db8856a3d0244d591e8d266ed4c38d8', 'request_id': 'req-7ed2e20f-8a18-46c9-aabd-608e182220c4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.683286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] The token cache is being invalidate because project 8db8856a3d0244d591e8d266ed4c38d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:41.684474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.685234 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7ed2e20f-8a18-46c9-aabd-608e182220c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:41.686917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1392/2772] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:41 2022] DELETE /identity/v3/projects/8db8856a3d0244d591e8d266ed4c38d8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:41.688319 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4302789-0038-4ece-89fa-9594d12dd691 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:41.692737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddc350-9314-433d-ab72-ac3b7444a817 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:41.693061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddc350-9314-433d-ab72-ac3b7444a817 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:41.693377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddc350-9314-433d-ab72-ac3b7444a817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:41.719195 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-6bddc350-9314-433d-ab72-ac3b7444a817 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:41.720852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6bddc350-9314-433d-ab72-ac3b7444a817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:41.747851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257e719-de89-4c31-a77c-1801c47acdda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:41.748484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257e719-de89-4c31-a77c-1801c47acdda service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:41.748760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257e719-de89-4c31-a77c-1801c47acdda service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:41.749170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257e719-de89-4c31-a77c-1801c47acdda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:41.751002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4257e719-de89-4c31-a77c-1801c47acdda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:41.786006 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1393/2773] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:41.794538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47da70c-42db-4577-a25a-c26c91b37128 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:41.794825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47da70c-42db-4577-a25a-c26c91b37128 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:41.795125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47da70c-42db-4577-a25a-c26c91b37128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:41.818607 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257e719-de89-4c31-a77c-1801c47acdda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:41.819491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d47da70c-42db-4577-a25a-c26c91b37128 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:41.820101 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257e719-de89-4c31-a77c-1801c47acdda service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:41.821059 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d47da70c-42db-4577-a25a-c26c91b37128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:41.825264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:41.829320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1394/2774] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:41.831268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1381/2775] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:41.833380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47da70c-42db-4577-a25a-c26c91b37128 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:41.843985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47da70c-42db-4577-a25a-c26c91b37128 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:41.848676 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175466f-8d9a-484e-90c1-366c36a572ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:41.849315 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175466f-8d9a-484e-90c1-366c36a572ee admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:41.849594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175466f-8d9a-484e-90c1-366c36a572ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:41.849860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175466f-8d9a-484e-90c1-366c36a572ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:41.850531 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175466f-8d9a-484e-90c1-366c36a572ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:41.850828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175466f-8d9a-484e-90c1-366c36a572ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:41.853046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175466f-8d9a-484e-90c1-366c36a572ee admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:41.864952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1395/2776] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:41.869434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257e719-de89-4c31-a77c-1801c47acdda service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:41.879659 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257e719-de89-4c31-a77c-1801c47acdda service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:41.885004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0717082-64e4-49a7-b636-a88db205a0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:41.885610 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0717082-64e4-49a7-b636-a88db205a0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:41.885890 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0717082-64e4-49a7-b636-a88db205a0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:41.886165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0717082-64e4-49a7-b636-a88db205a0a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:41.886799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0717082-64e4-49a7-b636-a88db205a0a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:41.888829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0717082-64e4-49a7-b636-a88db205a0a6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:41.918864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1382/2777] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:41 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:41.923051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175466f-8d9a-484e-90c1-366c36a572ee admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:41.934061 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175466f-8d9a-484e-90c1-366c36a572ee admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:41.999830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a982a2-36b4-4400-a8ce-4997b27969bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.000573 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a982a2-36b4-4400-a8ce-4997b27969bf admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.000767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a982a2-36b4-4400-a8ce-4997b27969bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.001031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a982a2-36b4-4400-a8ce-4997b27969bf admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.001672 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a982a2-36b4-4400-a8ce-4997b27969bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.004129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a982a2-36b4-4400-a8ce-4997b27969bf admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.031469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1396/2778] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:41 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:42.035879 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0717082-64e4-49a7-b636-a88db205a0a6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.048948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0717082-64e4-49a7-b636-a88db205a0a6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.053940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abfc43-7db9-4c74-aa38-3b9be99fa55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.054611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abfc43-7db9-4c74-aa38-3b9be99fa55b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.054902 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abfc43-7db9-4c74-aa38-3b9be99fa55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.055224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abfc43-7db9-4c74-aa38-3b9be99fa55b admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.055899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abfc43-7db9-4c74-aa38-3b9be99fa55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.057794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abfc43-7db9-4c74-aa38-3b9be99fa55b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.075406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1383/2779] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:42.077523 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a982a2-36b4-4400-a8ce-4997b27969bf admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.089589 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a982a2-36b4-4400-a8ce-4997b27969bf admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.095207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b8a3-1a7c-45f3-8037-d946ec0e8eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.095834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b8a3-1a7c-45f3-8037-d946ec0e8eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.096094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b8a3-1a7c-45f3-8037-d946ec0e8eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.096363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b8a3-1a7c-45f3-8037-d946ec0e8eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.097106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2b8a3-1a7c-45f3-8037-d946ec0e8eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.098752 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2b8a3-1a7c-45f3-8037-d946ec0e8eb8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.110960 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1397/2780] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:42.115408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abfc43-7db9-4c74-aa38-3b9be99fa55b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.127242 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abfc43-7db9-4c74-aa38-3b9be99fa55b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.127609 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b8a3-1a7c-45f3-8037-d946ec0e8eb8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.129509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-35e2b8a3-1a7c-45f3-8037-d946ec0e8eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.135087 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac001a-f794-4655-9ece-15e37b3098e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.135758 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac001a-f794-4655-9ece-15e37b3098e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.136170 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac001a-f794-4655-9ece-15e37b3098e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.136447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac001a-f794-4655-9ece-15e37b3098e7 admin admin] PATH_INFO: `/v3/projects/bab40b6a79c7444a8f6c4466cca1bdf7/users/20f3464822154f3c9ce98220b001b54a/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.145933 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b8a3-1a7c-45f3-8037-d946ec0e8eb8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.151802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.152985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.153292 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.153661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] PATH_INFO: `/v3/projects/11d5687cea724eba885f31c6693de8fd` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.154895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d5687cea724eba885f31c6693de8fd)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.157071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.165064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac001a-f794-4655-9ece-15e37b3098e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab40b6a79c7444a8f6c4466cca1bdf7, user_id=20f3464822154f3c9ce98220b001b54a, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.167850 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac001a-f794-4655-9ece-15e37b3098e7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.185349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1384/2781] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:42 2022] PUT /identity/v3/projects/bab40b6a79c7444a8f6c4466cca1bdf7/users/20f3464822154f3c9ce98220b001b54a/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:42.193377 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:42.196381 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a23394-ccbe-43fb-bb9e-59c15b24177c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.196719 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a23394-ccbe-43fb-bb9e-59c15b24177c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.197024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a23394-ccbe-43fb-bb9e-59c15b24177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.218685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d5687cea724eba885f31c6693de8fd', 'request_id': 'req-9ee35536-bbd7-4875-8d0d-3ac0650cb281'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.224337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d5687cea724eba885f31c6693de8fd', 'request_id': 'req-9ee35536-bbd7-4875-8d0d-3ac0650cb281'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.226721 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-14a23394-ccbe-43fb-bb9e-59c15b24177c None None] MFA Rules not processed for user `20f3464822154f3c9ce98220b001b54a`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:42.228318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-14a23394-ccbe-43fb-bb9e-59c15b24177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.232748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d5687cea724eba885f31c6693de8fd', 'request_id': 'req-9ee35536-bbd7-4875-8d0d-3ac0650cb281'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.244794 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] The token cache is being invalidate because project 11d5687cea724eba885f31c6693de8fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:42.245033 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.245683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.247157 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1398/2782] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:42 2022] DELETE /identity/v3/projects/11d5687cea724eba885f31c6693de8fd => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:42.250909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.252316 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.298397 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1385/2783] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:42.310013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a23394-ccbe-43fb-bb9e-59c15b24177c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.311546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-14a23394-ccbe-43fb-bb9e-59c15b24177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.321892 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee35536-bbd7-4875-8d0d-3ac0650cb281 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.326538 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.327166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.327443 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.327710 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] PATH_INFO: `/v3/users/8f5b92ae97c54be7980f4298b1f0ae6c` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.328856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f5b92ae97c54be7980f4298b1f0ae6c)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.330797 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.371148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a23394-ccbe-43fb-bb9e-59c15b24177c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.374864 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d87f87-c6c5-4792-a49f-8a4bea78bd52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.375472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d87f87-c6c5-4792-a49f-8a4bea78bd52 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.375952 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d87f87-c6c5-4792-a49f-8a4bea78bd52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.376071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d87f87-c6c5-4792-a49f-8a4bea78bd52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.378511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f5b92ae97c54be7980f4298b1f0ae6c'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.379145 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f5b92ae97c54be7980f4298b1f0ae6c'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.385701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d87f87-c6c5-4792-a49f-8a4bea78bd52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.386966 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f5b92ae97c54be7980f4298b1f0ae6c'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.387145 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d87f87-c6c5-4792-a49f-8a4bea78bd52 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.387736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f5b92ae97c54be7980f4298b1f0ae6c'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.394852 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6a99b93b-c5ae-46f6-8ff1-b4c08f4c5767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f5b92ae97c54be7980f4298b1f0ae6c'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.396320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1386/2784] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:42.413453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1399/2785] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:42 2022] DELETE /identity/v3/users/8f5b92ae97c54be7980f4298b1f0ae6c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:42.415402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d87f87-c6c5-4792-a49f-8a4bea78bd52 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.416836 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d8d87f87-c6c5-4792-a49f-8a4bea78bd52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.432095 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d87f87-c6c5-4792-a49f-8a4bea78bd52 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.467807 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:42.496999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.497563 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.497851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.498123 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] PATH_INFO: `/v3/users/20f3464822154f3c9ce98220b001b54a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.499257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f3464822154f3c9ce98220b001b54a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.501223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.555652 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f3464822154f3c9ce98220b001b54a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.566339 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f3464822154f3c9ce98220b001b54a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.567466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f3464822154f3c9ce98220b001b54a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.576648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f3464822154f3c9ce98220b001b54a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.577971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f3464822154f3c9ce98220b001b54a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.596503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1387/2786] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:42 2022] DELETE /identity/v3/users/20f3464822154f3c9ce98220b001b54a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:42.600983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd9288-e7c6-4dc4-b383-b692bb2f5b16 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.601973 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd9288-e7c6-4dc4-b383-b692bb2f5b16 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.602489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd9288-e7c6-4dc4-b383-b692bb2f5b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.622345 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d9fd9288-e7c6-4dc4-b383-b692bb2f5b16 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:42.623579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d9fd9288-e7c6-4dc4-b383-b692bb2f5b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.652094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:42.693276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1400/2787] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:42.698004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.699412 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.708618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd9288-e7c6-4dc4-b383-b692bb2f5b16 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.712776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d9fd9288-e7c6-4dc4-b383-b692bb2f5b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.716635 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa8a34-ef09-4c57-884f-306c9e2ae016 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.721553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.722349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.722638 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.722965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] PATH_INFO: `/v3/projects/7263fe03e82747ff9ba799f88a0debfd` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.724071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7263fe03e82747ff9ba799f88a0debfd)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.726445 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.793487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7263fe03e82747ff9ba799f88a0debfd', 'request_id': 'req-956f13d4-f38e-4ce8-857d-83acd482509b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.803000 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd9288-e7c6-4dc4-b383-b692bb2f5b16 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.804310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7263fe03e82747ff9ba799f88a0debfd', 'request_id': 'req-956f13d4-f38e-4ce8-857d-83acd482509b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.815891 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7263fe03e82747ff9ba799f88a0debfd', 'request_id': 'req-956f13d4-f38e-4ce8-857d-83acd482509b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.834932 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] The token cache is being invalidate because project 7263fe03e82747ff9ba799f88a0debfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:42.835347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.836724 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.842838 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1388/2788] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:42 2022] DELETE /identity/v3/projects/7263fe03e82747ff9ba799f88a0debfd => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:42.843985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:42.846471 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.886135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb1153-7dde-4c1b-9dca-7990f0a4f365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.886751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb1153-7dde-4c1b-9dca-7990f0a4f365 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.887036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb1153-7dde-4c1b-9dca-7990f0a4f365 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.887320 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb1153-7dde-4c1b-9dca-7990f0a4f365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.889130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f3cb1153-7dde-4c1b-9dca-7990f0a4f365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:42.904266 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:42.934706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f13d4-f38e-4ce8-857d-83acd482509b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:42.939414 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:42.940042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.940322 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.940592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] PATH_INFO: `/v3/users/2efc5ec2a3aa449dbc389dfd6766d030` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.941741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2efc5ec2a3aa449dbc389dfd6766d030)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.943685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.960199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb1153-7dde-4c1b-9dca-7990f0a4f365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:42.961940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb1153-7dde-4c1b-9dca-7990f0a4f365 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:42.972028 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1401/2789] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:42.973287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2759a-44fc-4b59-862f-01b28918e3d5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:42.973572 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2759a-44fc-4b59-862f-01b28918e3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:42.973932 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2759a-44fc-4b59-862f-01b28918e3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:42.995198 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-01f2759a-44fc-4b59-862f-01b28918e3d5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:42.996298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2efc5ec2a3aa449dbc389dfd6766d030'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:42.996445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-01f2759a-44fc-4b59-862f-01b28918e3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.006148 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1402/2790] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:43.009502 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2efc5ec2a3aa449dbc389dfd6766d030'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.010377 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2efc5ec2a3aa449dbc389dfd6766d030'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.014073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fdee6-c646-4c94-afcf-16d8288f9cb5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.014351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fdee6-c646-4c94-afcf-16d8288f9cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.014614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fdee6-c646-4c94-afcf-16d8288f9cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.022816 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2efc5ec2a3aa449dbc389dfd6766d030'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.024390 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c801afcb-1f2b-46b8-b83a-f16f0c57ddc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2efc5ec2a3aa449dbc389dfd6766d030'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.035614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-c47fdee6-c646-4c94-afcf-16d8288f9cb5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:43.036842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c47fdee6-c646-4c94-afcf-16d8288f9cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.044379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1389/2791] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:42 2022] DELETE /identity/v3/users/2efc5ec2a3aa449dbc389dfd6766d030 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:43.045751 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1403/2792] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:43.049569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fdee6-c646-4c94-afcf-16d8288f9cb5 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.050211 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd6647-481c-44cf-b18a-ecffbdbe6b64 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.050495 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd6647-481c-44cf-b18a-ecffbdbe6b64 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.050771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd6647-481c-44cf-b18a-ecffbdbe6b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.060608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fdee6-c646-4c94-afcf-16d8288f9cb5 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.074166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1ecd6647-481c-44cf-b18a-ecffbdbe6b64 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:43.075591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1ecd6647-481c-44cf-b18a-ecffbdbe6b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.084904 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:43.135701 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.136354 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.136834 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.136954 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.137653 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:43.137921 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.139467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.153771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1390/2793] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:43.155569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1404/2794] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:43.159685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd6647-481c-44cf-b18a-ecffbdbe6b64 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.165817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.170078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.173127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd6647-481c-44cf-b18a-ecffbdbe6b64 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.179033 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8f4f5-3f3a-4f59-9f9e-031cfae6d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.180349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8f4f5-3f3a-4f59-9f9e-031cfae6d54a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.180633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8f4f5-3f3a-4f59-9f9e-031cfae6d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.181729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8f4f5-3f3a-4f59-9f9e-031cfae6d54a admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.182518 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8f4f5-3f3a-4f59-9f9e-031cfae6d54a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.185027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8f4f5-3f3a-4f59-9f9e-031cfae6d54a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.222943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1391/2795] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:43.227928 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8f4f5-3f3a-4f59-9f9e-031cfae6d54a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.248765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b988df-3f2c-41ce-afc9-16aa9b6a93d4 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.276292 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:43.307655 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8f4f5-3f3a-4f59-9f9e-031cfae6d54a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.311519 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70902e-1bd2-46a6-b804-4badaa2be601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.312138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70902e-1bd2-46a6-b804-4badaa2be601 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.312456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70902e-1bd2-46a6-b804-4badaa2be601 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.312835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70902e-1bd2-46a6-b804-4badaa2be601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.314234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c02b7-b5a5-4b74-83c3-3e172112ed5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.315001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c02b7-b5a5-4b74-83c3-3e172112ed5e admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.315379 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c02b7-b5a5-4b74-83c3-3e172112ed5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.315773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c02b7-b5a5-4b74-83c3-3e172112ed5e admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.316490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c02b7-b5a5-4b74-83c3-3e172112ed5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.319314 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c02b7-b5a5-4b74-83c3-3e172112ed5e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.323522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70902e-1bd2-46a6-b804-4badaa2be601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.325079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70902e-1bd2-46a6-b804-4badaa2be601 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.336561 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1405/2796] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:43.337790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70902e-1bd2-46a6-b804-4badaa2be601 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.339507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7d70902e-1bd2-46a6-b804-4badaa2be601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.345355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1392/2797] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:43.349699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c02b7-b5a5-4b74-83c3-3e172112ed5e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.353189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70902e-1bd2-46a6-b804-4badaa2be601 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.358604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.359358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.359654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.359896 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c02b7-b5a5-4b74-83c3-3e172112ed5e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.360021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] PATH_INFO: `/v3/projects/bab40b6a79c7444a8f6c4466cca1bdf7` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.361074 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab40b6a79c7444a8f6c4466cca1bdf7)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.363114 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.365049 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea1064-cd1c-4532-9916-6ca8d9c9edaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.365704 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea1064-cd1c-4532-9916-6ca8d9c9edaf admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.366007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea1064-cd1c-4532-9916-6ca8d9c9edaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.366278 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea1064-cd1c-4532-9916-6ca8d9c9edaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.367480 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea1064-cd1c-4532-9916-6ca8d9c9edaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.369113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea1064-cd1c-4532-9916-6ca8d9c9edaf admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.380506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1393/2798] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:43.385517 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea1064-cd1c-4532-9916-6ca8d9c9edaf admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.398760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea1064-cd1c-4532-9916-6ca8d9c9edaf admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.403700 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ae7b3-2d70-42db-9328-ae90a2548c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.404606 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ae7b3-2d70-42db-9328-ae90a2548c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.404923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ae7b3-2d70-42db-9328-ae90a2548c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.405204 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ae7b3-2d70-42db-9328-ae90a2548c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.405848 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ae7b3-2d70-42db-9328-ae90a2548c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.407633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ae7b3-2d70-42db-9328-ae90a2548c6a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.420557 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab40b6a79c7444a8f6c4466cca1bdf7', 'request_id': 'req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.424632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1394/2799] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:43.428983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ae7b3-2d70-42db-9328-ae90a2548c6a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.432543 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab40b6a79c7444a8f6c4466cca1bdf7', 'request_id': 'req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.441040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ae7b3-2d70-42db-9328-ae90a2548c6a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.442428 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab40b6a79c7444a8f6c4466cca1bdf7', 'request_id': 'req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.452130 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:43.457093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] The token cache is being invalidate because project bab40b6a79c7444a8f6c4466cca1bdf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:43.457403 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.458190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-01c722fe-6bb7-47d2-ac17-7bf5c56adff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.460865 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1406/2800] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:43 2022] DELETE /identity/v3/projects/bab40b6a79c7444a8f6c4466cca1bdf7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:43.466177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f592-beb1-49dd-aa53-f0c95a29d5ae None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.466958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f592-beb1-49dd-aa53-f0c95a29d5ae None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.467539 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f592-beb1-49dd-aa53-f0c95a29d5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.489445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d0e4f592-beb1-49dd-aa53-f0c95a29d5ae None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:43.490945 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d0e4f592-beb1-49dd-aa53-f0c95a29d5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.515589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c89bb-6049-4586-9454-1ed6105d9767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.516918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c89bb-6049-4586-9454-1ed6105d9767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.517203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c89bb-6049-4586-9454-1ed6105d9767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.517483 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c89bb-6049-4586-9454-1ed6105d9767 admin admin] PATH_INFO: `/v3/projects/45a9d11f80d64ea2b9ba6a32b06320e4/users/964301bb43684a25b0cc10855faf6cae/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.534732 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c89bb-6049-4586-9454-1ed6105d9767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a9d11f80d64ea2b9ba6a32b06320e4, user_id=964301bb43684a25b0cc10855faf6cae, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.537416 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c89bb-6049-4586-9454-1ed6105d9767 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.560781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1395/2801] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:43 2022] PUT /identity/v3/projects/45a9d11f80d64ea2b9ba6a32b06320e4/users/964301bb43684a25b0cc10855faf6cae/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:43.567254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1407/2802] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:43.571711 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3743a33-323e-44d5-92fe-c267188af152 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.571996 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3743a33-323e-44d5-92fe-c267188af152 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.572264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3743a33-323e-44d5-92fe-c267188af152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.575435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8875e-2490-48fd-b0db-fa5daedfbc36 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.575725 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8875e-2490-48fd-b0db-fa5daedfbc36 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.575998 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8875e-2490-48fd-b0db-fa5daedfbc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.598567 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-17b8875e-2490-48fd-b0db-fa5daedfbc36 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:43.600238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-17b8875e-2490-48fd-b0db-fa5daedfbc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.602714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b3743a33-323e-44d5-92fe-c267188af152 None None] MFA Rules not processed for user `964301bb43684a25b0cc10855faf6cae`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:43.604155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b3743a33-323e-44d5-92fe-c267188af152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.668501 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1408/2803] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:43.669566 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8875e-2490-48fd-b0db-fa5daedfbc36 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.671376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-17b8875e-2490-48fd-b0db-fa5daedfbc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.685290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1396/2804] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:43.686153 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3743a33-323e-44d5-92fe-c267188af152 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.691992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8875e-2490-48fd-b0db-fa5daedfbc36 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.698482 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.699118 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.699398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.699661 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] PATH_INFO: `/v3/projects/809367e544b54add8b364e3568d2bcc1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.700835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=809367e544b54add8b364e3568d2bcc1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.701616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3743a33-323e-44d5-92fe-c267188af152 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.702823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.706531 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.707210 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.707550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.707899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.708603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:43.709003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.711345 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.723097 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1397/2805] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:43.723907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.725739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.731473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:43.753687 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '809367e544b54add8b364e3568d2bcc1', 'request_id': 'req-06764a23-2e91-4ffe-8268-b8f9953733e8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.761057 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '809367e544b54add8b364e3568d2bcc1', 'request_id': 'req-06764a23-2e91-4ffe-8268-b8f9953733e8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.770160 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '809367e544b54add8b364e3568d2bcc1', 'request_id': 'req-06764a23-2e91-4ffe-8268-b8f9953733e8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.782378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] The token cache is being invalidate because project 809367e544b54add8b364e3568d2bcc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:43.782679 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.783332 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:43.784933 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1409/2806] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:43 2022] DELETE /identity/v3/projects/809367e544b54add8b364e3568d2bcc1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:43.785705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.787002 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.787931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37871ab3-cb7d-4937-b456-18980987a8f0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.817161 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:43.840127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c982c-3c12-4459-b982-b426c1bde303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.840754 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c982c-3c12-4459-b982-b426c1bde303 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.841049 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c982c-3c12-4459-b982-b426c1bde303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.841347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c982c-3c12-4459-b982-b426c1bde303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.843165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a84c982c-3c12-4459-b982-b426c1bde303 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.858236 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06764a23-2e91-4ffe-8268-b8f9953733e8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:43.863357 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fd442-11f9-46d4-8940-740d60c3fec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:43.863999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fd442-11f9-46d4-8940-740d60c3fec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.864281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fd442-11f9-46d4-8940-740d60c3fec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.864554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fd442-11f9-46d4-8940-740d60c3fec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.865233 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fd442-11f9-46d4-8940-740d60c3fec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.868197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fd442-11f9-46d4-8940-740d60c3fec7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.901093 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1410/2807] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:43.902104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84302f72-4534-4d04-826a-543c294559f8 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:43.902398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84302f72-4534-4d04-826a-543c294559f8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:43.902724 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84302f72-4534-4d04-826a-543c294559f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:43.906159 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c982c-3c12-4459-b982-b426c1bde303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:43.907765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c982c-3c12-4459-b982-b426c1bde303 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:43.923848 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-84302f72-4534-4d04-826a-543c294559f8 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:43.924933 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1398/2808] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:43.925184 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-84302f72-4534-4d04-826a-543c294559f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:43.926600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c982c-3c12-4459-b982-b426c1bde303 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:43.948447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c982c-3c12-4459-b982-b426c1bde303 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.017056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1411/2809] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:44.018863 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ec2a9-6544-4b94-a9e2-0132d50b03f5 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.019269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ec2a9-6544-4b94-a9e2-0132d50b03f5 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.019432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ec2a9-6544-4b94-a9e2-0132d50b03f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.019588 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3317b5a-7553-4bfc-b4ca-3de7e7c4712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.020192 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3317b5a-7553-4bfc-b4ca-3de7e7c4712d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.020468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3317b5a-7553-4bfc-b4ca-3de7e7c4712d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.020737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3317b5a-7553-4bfc-b4ca-3de7e7c4712d admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.021389 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3317b5a-7553-4bfc-b4ca-3de7e7c4712d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.023387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3317b5a-7553-4bfc-b4ca-3de7e7c4712d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.043776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-b26ec2a9-6544-4b94-a9e2-0132d50b03f5 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:44.045082 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b26ec2a9-6544-4b94-a9e2-0132d50b03f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:44.051285 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1399/2810] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:43 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:44.056290 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3317b5a-7553-4bfc-b4ca-3de7e7c4712d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.057056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1412/2811] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:44.061683 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ec2a9-6544-4b94-a9e2-0132d50b03f5 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.062804 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:44.071627 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3317b5a-7553-4bfc-b4ca-3de7e7c4712d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.076349 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ec2a9-6544-4b94-a9e2-0132d50b03f5 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.077535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61abe7-6f89-4a72-89a5-a6d8cad4de18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.078347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61abe7-6f89-4a72-89a5-a6d8cad4de18 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.078694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61abe7-6f89-4a72-89a5-a6d8cad4de18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.079130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61abe7-6f89-4a72-89a5-a6d8cad4de18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.080041 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61abe7-6f89-4a72-89a5-a6d8cad4de18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.083160 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61abe7-6f89-4a72-89a5-a6d8cad4de18 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.086360 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e8e2-6ef5-441b-bee0-c782b0f6d8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.088262 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e8e2-6ef5-441b-bee0-c782b0f6d8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.088551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e8e2-6ef5-441b-bee0-c782b0f6d8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.088995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e8e2-6ef5-441b-bee0-c782b0f6d8e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.090440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77e8e2-6ef5-441b-bee0-c782b0f6d8e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:44.090754 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77e8e2-6ef5-441b-bee0-c782b0f6d8e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.093472 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77e8e2-6ef5-441b-bee0-c782b0f6d8e2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.094065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1400/2812] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:44.098739 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61abe7-6f89-4a72-89a5-a6d8cad4de18 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.112283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61abe7-6f89-4a72-89a5-a6d8cad4de18 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.115398 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1413/2813] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 11:32:44.120451 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e8e2-6ef5-441b-bee0-c782b0f6d8e2 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.135822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aedd8-d2e3-4c5c-94fd-d79c7f1a007b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.136510 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aedd8-d2e3-4c5c-94fd-d79c7f1a007b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.136841 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aedd8-d2e3-4c5c-94fd-d79c7f1a007b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.137180 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aedd8-d2e3-4c5c-94fd-d79c7f1a007b admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.137805 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72aedd8-d2e3-4c5c-94fd-d79c7f1a007b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.140126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72aedd8-d2e3-4c5c-94fd-d79c7f1a007b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.141452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e8e2-6ef5-441b-bee0-c782b0f6d8e2 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.147594 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319199e-1bcc-4301-b108-069f3cafb242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.148250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319199e-1bcc-4301-b108-069f3cafb242 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.148729 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319199e-1bcc-4301-b108-069f3cafb242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.149250 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319199e-1bcc-4301-b108-069f3cafb242 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.149897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319199e-1bcc-4301-b108-069f3cafb242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.151885 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319199e-1bcc-4301-b108-069f3cafb242 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.152594 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1401/2814] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:44.157238 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aedd8-d2e3-4c5c-94fd-d79c7f1a007b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.169766 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aedd8-d2e3-4c5c-94fd-d79c7f1a007b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.175054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9961e25-16aa-41f2-b3ad-55efa1328342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.175641 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9961e25-16aa-41f2-b3ad-55efa1328342 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.175948 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9961e25-16aa-41f2-b3ad-55efa1328342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.176223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9961e25-16aa-41f2-b3ad-55efa1328342 admin admin] PATH_INFO: `/v3/projects/0e986dde085e45649d1e85137af83b67/users/0edf928fb6d94e8e85f303ecaa3cad7a/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.191906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1414/2815] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:44 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:44.194142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9961e25-16aa-41f2-b3ad-55efa1328342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e986dde085e45649d1e85137af83b67, user_id=0edf928fb6d94e8e85f303ecaa3cad7a, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.195990 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319199e-1bcc-4301-b108-069f3cafb242 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.201384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9961e25-16aa-41f2-b3ad-55efa1328342 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.225498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1402/2816] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:44 2022] PUT /identity/v3/projects/0e986dde085e45649d1e85137af83b67/users/0edf928fb6d94e8e85f303ecaa3cad7a/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:44.228926 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5f732-988e-4092-ae56-fe306625e506 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.229187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5f732-988e-4092-ae56-fe306625e506 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.229354 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5f732-988e-4092-ae56-fe306625e506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.269420 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-76f5f732-988e-4092-ae56-fe306625e506 None None] MFA Rules not processed for user `0edf928fb6d94e8e85f303ecaa3cad7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:44.270647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-76f5f732-988e-4092-ae56-fe306625e506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:44.277771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319199e-1bcc-4301-b108-069f3cafb242 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.291757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:44.355273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1403/2817] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:44.363922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fd6d9-994d-478c-9f1a-58be0ce8c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.364664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fd6d9-994d-478c-9f1a-58be0ce8c080 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.365051 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fd6d9-994d-478c-9f1a-58be0ce8c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.365402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fd6d9-994d-478c-9f1a-58be0ce8c080 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.366166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fd6d9-994d-478c-9f1a-58be0ce8c080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.367874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5f732-988e-4092-ae56-fe306625e506 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.368686 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fd6d9-994d-478c-9f1a-58be0ce8c080 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.370028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-76f5f732-988e-4092-ae56-fe306625e506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:44.391612 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1415/2818] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:44 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:44.395820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fd6d9-994d-478c-9f1a-58be0ce8c080 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.406618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fd6d9-994d-478c-9f1a-58be0ce8c080 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.411276 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dd831-798a-45b3-8f88-2309dc4b4a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.411872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dd831-798a-45b3-8f88-2309dc4b4a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.412153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dd831-798a-45b3-8f88-2309dc4b4a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.412430 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dd831-798a-45b3-8f88-2309dc4b4a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.413101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92dd831-798a-45b3-8f88-2309dc4b4a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.414768 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92dd831-798a-45b3-8f88-2309dc4b4a34 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.425077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1416/2819] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:44.428785 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dd831-798a-45b3-8f88-2309dc4b4a34 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.435523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5f732-988e-4092-ae56-fe306625e506 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.439308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dd831-798a-45b3-8f88-2309dc4b4a34 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.439578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ade5c6-bbd0-41af-a666-ebedcc913067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.440187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ade5c6-bbd0-41af-a666-ebedcc913067 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.440468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ade5c6-bbd0-41af-a666-ebedcc913067 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.440746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ade5c6-bbd0-41af-a666-ebedcc913067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.443974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58218d-cdaf-4c9d-bc19-2a9fcd3ed658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.444564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58218d-cdaf-4c9d-bc19-2a9fcd3ed658 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.445490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58218d-cdaf-4c9d-bc19-2a9fcd3ed658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.445816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58218d-cdaf-4c9d-bc19-2a9fcd3ed658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.446598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58218d-cdaf-4c9d-bc19-2a9fcd3ed658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.448211 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58218d-cdaf-4c9d-bc19-2a9fcd3ed658 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.450310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ade5c6-bbd0-41af-a666-ebedcc913067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.452165 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ade5c6-bbd0-41af-a666-ebedcc913067 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.461494 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1417/2820] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:44.462171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1404/2821] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:44.465614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58218d-cdaf-4c9d-bc19-2a9fcd3ed658 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.475666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58218d-cdaf-4c9d-bc19-2a9fcd3ed658 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.481767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741f29d-1e9b-4042-b4c1-d29fa4ef81c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.482544 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741f29d-1e9b-4042-b4c1-d29fa4ef81c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.482967 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741f29d-1e9b-4042-b4c1-d29fa4ef81c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.483246 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741f29d-1e9b-4042-b4c1-d29fa4ef81c8 admin admin] PATH_INFO: `/v3/projects/5bebc3459c1c4e5f8e766d90f0859b16/users/53d8bbb16c794d13bf005e21b832c87e/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.501319 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741f29d-1e9b-4042-b4c1-d29fa4ef81c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bebc3459c1c4e5f8e766d90f0859b16, user_id=53d8bbb16c794d13bf005e21b832c87e, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.504204 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741f29d-1e9b-4042-b4c1-d29fa4ef81c8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.520619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1418/2822] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:44 2022] PUT /identity/v3/projects/5bebc3459c1c4e5f8e766d90f0859b16/users/53d8bbb16c794d13bf005e21b832c87e/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:44.526799 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae453-7347-4259-b9db-92e123bbb7ed None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.527281 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae453-7347-4259-b9db-92e123bbb7ed None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.527376 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae453-7347-4259-b9db-92e123bbb7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.552463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3ccae453-7347-4259-b9db-92e123bbb7ed None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:44.553746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3ccae453-7347-4259-b9db-92e123bbb7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:44.601036 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:44.622813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1405/2823] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:44.632599 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741f29d-1e9b-4042-b4c1-d29fa4ef81c8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:44.634600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7741f29d-1e9b-4042-b4c1-d29fa4ef81c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:44.697861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741f29d-1e9b-4042-b4c1-d29fa4ef81c8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:44.701682 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8e413-27b2-4e5c-877b-546607ae4913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:44.702213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8e413-27b2-4e5c-877b-546607ae4913 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:44.702819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8e413-27b2-4e5c-877b-546607ae4913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:44.702819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8e413-27b2-4e5c-877b-546607ae4913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:44.713128 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8e413-27b2-4e5c-877b-546607ae4913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:44.714590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8e413-27b2-4e5c-877b-546607ae4913 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:44.731842 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1419/2824] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:45.238382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccae453-7347-4259-b9db-92e123bbb7ed None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:45.248771 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccae453-7347-4259-b9db-92e123bbb7ed None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:45.252548 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62efd14-2d10-4795-b305-04d57941c6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:45.253283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62efd14-2d10-4795-b305-04d57941c6eb service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:45.253661 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62efd14-2d10-4795-b305-04d57941c6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:45.253913 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62efd14-2d10-4795-b305-04d57941c6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:45.256930 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a62efd14-2d10-4795-b305-04d57941c6eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:45.276157 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62efd14-2d10-4795-b305-04d57941c6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:45.277647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62efd14-2d10-4795-b305-04d57941c6eb service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:45.287653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1406/2825] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:50.575942 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8e413-27b2-4e5c-877b-546607ae4913 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:50.577778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-eca8e413-27b2-4e5c-877b-546607ae4913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:50.594849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8e413-27b2-4e5c-877b-546607ae4913 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:50.601029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:50.601802 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:50.602063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:50.602337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] PATH_INFO: `/v3/users/e62100a950db47738a7f0e3186e2344a` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:50.603480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e62100a950db47738a7f0e3186e2344a)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:50.605956 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:50.672792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e62100a950db47738a7f0e3186e2344a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:50.673487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e62100a950db47738a7f0e3186e2344a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:50.684054 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e62100a950db47738a7f0e3186e2344a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:50.684762 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e62100a950db47738a7f0e3186e2344a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:50.691131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:50.693269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e62100a950db47738a7f0e3186e2344a'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:50.728335 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1420/2826] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:50 2022] DELETE /identity/v3/users/e62100a950db47738a7f0e3186e2344a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:50.932876 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62efd14-2d10-4795-b305-04d57941c6eb service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:50.946109 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62efd14-2d10-4795-b305-04d57941c6eb service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.094512 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.095148 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.095382 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.095730 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] PATH_INFO: `/v3/users/0edf928fb6d94e8e85f303ecaa3cad7a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.106337 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.107316 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0edf928fb6d94e8e85f303ecaa3cad7a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.107993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:51.114916 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.135047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126e43d-eb6b-47d0-9891-56b135024f45 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.142709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.143563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.145697 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.146076 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] PATH_INFO: `/v3/projects/9bad2083e5cf40c5bf1f853130262e64` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.148651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bad2083e5cf40c5bf1f853130262e64)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.151600 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.190687 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0edf928fb6d94e8e85f303ecaa3cad7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.201632 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0edf928fb6d94e8e85f303ecaa3cad7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.202492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0edf928fb6d94e8e85f303ecaa3cad7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.217233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0edf928fb6d94e8e85f303ecaa3cad7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.218220 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-daa96e12-93b4-4183-ae02-7ccb159be570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0edf928fb6d94e8e85f303ecaa3cad7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.251812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1407/2827] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:50 2022] DELETE /identity/v3/users/0edf928fb6d94e8e85f303ecaa3cad7a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:51.253455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6b3f5-5255-40f7-826f-2df99c214ded None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.253947 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6b3f5-5255-40f7-826f-2df99c214ded None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.254335 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6b3f5-5255-40f7-826f-2df99c214ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.257991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bad2083e5cf40c5bf1f853130262e64', 'request_id': 'req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.271488 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bad2083e5cf40c5bf1f853130262e64', 'request_id': 'req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.285299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bad2083e5cf40c5bf1f853130262e64', 'request_id': 'req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.293535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7af6b3f5-5255-40f7-826f-2df99c214ded None None] MFA Rules not processed for user `53d8bbb16c794d13bf005e21b832c87e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:51.294893 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7af6b3f5-5255-40f7-826f-2df99c214ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:51.325964 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] The token cache is being invalidate because project 9bad2083e5cf40c5bf1f853130262e64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:51.326271 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.328783 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-5d3d84bb-b2e0-4696-b1fb-9933cc79c5a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:51.332902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1421/2828] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:51 2022] DELETE /identity/v3/projects/9bad2083e5cf40c5bf1f853130262e64 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:51.334764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a61ab-509e-4047-b289-4afdc185c51c None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.336154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a61ab-509e-4047-b289-4afdc185c51c None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.337521 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a61ab-509e-4047-b289-4afdc185c51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.358968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:51.369016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-353a61ab-509e-4047-b289-4afdc185c51c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:51.374173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-353a61ab-509e-4047-b289-4afdc185c51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:51.394843 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1408/2829] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:51.396747 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa92be-3941-4d7f-91fa-7a328154ca9c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.397078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa92be-3941-4d7f-91fa-7a328154ca9c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.397353 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa92be-3941-4d7f-91fa-7a328154ca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.421250 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-91fa92be-3941-4d7f-91fa-7a328154ca9c None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:51.422630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-91fa92be-3941-4d7f-91fa-7a328154ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:51.468975 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1422/2830] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:51.469830 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a61ab-509e-4047-b289-4afdc185c51c None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.471119 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-353a61ab-509e-4047-b289-4afdc185c51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:51.484185 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a61ab-509e-4047-b289-4afdc185c51c None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.488846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4d321-c437-474a-98ac-34674f101529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.489520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4d321-c437-474a-98ac-34674f101529 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.489728 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4d321-c437-474a-98ac-34674f101529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.489987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4d321-c437-474a-98ac-34674f101529 admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.491064 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1409/2831] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:51.492350 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa92be-3941-4d7f-91fa-7a328154ca9c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.493774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-91fa92be-3941-4d7f-91fa-7a328154ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:51.497395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4d321-c437-474a-98ac-34674f101529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.499668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4d321-c437-474a-98ac-34674f101529 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.532667 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1423/2832] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:51 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:51.534113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723bdf-9d25-4a30-8d0f-763c1a27a70a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.534469 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723bdf-9d25-4a30-8d0f-763c1a27a70a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.535092 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723bdf-9d25-4a30-8d0f-763c1a27a70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.556008 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d8723bdf-9d25-4a30-8d0f-763c1a27a70a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:51.557273 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d8723bdf-9d25-4a30-8d0f-763c1a27a70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:51.570999 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa92be-3941-4d7f-91fa-7a328154ca9c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.589025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:51.621804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1424/2833] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:51.622121 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1b12e-9c95-4fb4-817f-d8464fea4e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.622702 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8723bdf-9d25-4a30-8d0f-763c1a27a70a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.622825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1b12e-9c95-4fb4-817f-d8464fea4e92 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.623009 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1b12e-9c95-4fb4-817f-d8464fea4e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.623391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1b12e-9c95-4fb4-817f-d8464fea4e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.632514 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8723bdf-9d25-4a30-8d0f-763c1a27a70a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.633519 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1b12e-9c95-4fb4-817f-d8464fea4e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.634899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1b12e-9c95-4fb4-817f-d8464fea4e92 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.637678 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90949533-d9c9-4e7c-8805-e34e961f5648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.638274 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90949533-d9c9-4e7c-8805-e34e961f5648 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.638551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90949533-d9c9-4e7c-8805-e34e961f5648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.638818 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90949533-d9c9-4e7c-8805-e34e961f5648 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.639435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90949533-d9c9-4e7c-8805-e34e961f5648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.641410 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90949533-d9c9-4e7c-8805-e34e961f5648 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.644707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1410/2834] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:51.645534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1b12e-9c95-4fb4-817f-d8464fea4e92 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.655405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1b12e-9c95-4fb4-817f-d8464fea4e92 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.661117 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e1f45-bfc4-4d49-9456-386a892eaa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.661800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e1f45-bfc4-4d49-9456-386a892eaa69 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.662169 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e1f45-bfc4-4d49-9456-386a892eaa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.662443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e1f45-bfc4-4d49-9456-386a892eaa69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.663078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e1f45-bfc4-4d49-9456-386a892eaa69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:32:51.663900 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e1f45-bfc4-4d49-9456-386a892eaa69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.665714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e1f45-bfc4-4d49-9456-386a892eaa69 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.669781 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1425/2835] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:51 2022] POST /identity/v3/users => generated 299 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:51.670882 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90949533-d9c9-4e7c-8805-e34e961f5648 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.678714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1411/2836] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:32:51.682313 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90949533-d9c9-4e7c-8805-e34e961f5648 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.683825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e1f45-bfc4-4d49-9456-386a892eaa69 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.687149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f4925-d670-4fdb-beb6-6c24ad6f4f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.687757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f4925-d670-4fdb-beb6-6c24ad6f4f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.688035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f4925-d670-4fdb-beb6-6c24ad6f4f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.688308 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f4925-d670-4fdb-beb6-6c24ad6f4f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.688993 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f4925-d670-4fdb-beb6-6c24ad6f4f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.690883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f4925-d670-4fdb-beb6-6c24ad6f4f19 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.696248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e1f45-bfc4-4d49-9456-386a892eaa69 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.702094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d72cf4-f947-484d-89a3-27fd8f011dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.702862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d72cf4-f947-484d-89a3-27fd8f011dff admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.703883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d72cf4-f947-484d-89a3-27fd8f011dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.704005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1426/2837] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:51.704116 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d72cf4-f947-484d-89a3-27fd8f011dff admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.704970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d72cf4-f947-484d-89a3-27fd8f011dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.707154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d72cf4-f947-484d-89a3-27fd8f011dff admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.708053 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f4925-d670-4fdb-beb6-6c24ad6f4f19 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.719688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f4925-d670-4fdb-beb6-6c24ad6f4f19 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.725154 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a5fa8-559a-4f4a-a791-d751f99295c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.725853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a5fa8-559a-4f4a-a791-d751f99295c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.726339 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a5fa8-559a-4f4a-a791-d751f99295c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.726486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a5fa8-559a-4f4a-a791-d751f99295c8 admin admin] PATH_INFO: `/v3/projects/23336ce1c1e74b7693d45ac090b2b935/users/7f7ec5c791134db082cbb362e56e92da/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.745586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1412/2838] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:32:51 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:32:51.746670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d72cf4-f947-484d-89a3-27fd8f011dff admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.747035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a5fa8-559a-4f4a-a791-d751f99295c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23336ce1c1e74b7693d45ac090b2b935, user_id=7f7ec5c791134db082cbb362e56e92da, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.749835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a5fa8-559a-4f4a-a791-d751f99295c8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.760400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:51.767950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1427/2839] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:51 2022] PUT /identity/v3/projects/23336ce1c1e74b7693d45ac090b2b935/users/7f7ec5c791134db082cbb362e56e92da/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:51.771651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a5fa8-559a-4f4a-a791-d751f99295c8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.844883 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d72cf4-f947-484d-89a3-27fd8f011dff admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.850490 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b9a8f-6418-48c8-851b-6327b8e3c1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.851664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b9a8f-6418-48c8-851b-6327b8e3c1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.852059 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b9a8f-6418-48c8-851b-6327b8e3c1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.852349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b9a8f-6418-48c8-851b-6327b8e3c1a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.852886 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a5fa8-559a-4f4a-a791-d751f99295c8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.853010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b9a8f-6418-48c8-851b-6327b8e3c1a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.855359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b9a8f-6418-48c8-851b-6327b8e3c1a7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.858362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77af6e-a50d-4cb9-8e80-1cdc01bc4f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.858985 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77af6e-a50d-4cb9-8e80-1cdc01bc4f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.859266 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77af6e-a50d-4cb9-8e80-1cdc01bc4f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.859536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77af6e-a50d-4cb9-8e80-1cdc01bc4f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.860173 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77af6e-a50d-4cb9-8e80-1cdc01bc4f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.861805 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77af6e-a50d-4cb9-8e80-1cdc01bc4f4f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.874474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1428/2840] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:51.878657 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77af6e-a50d-4cb9-8e80-1cdc01bc4f4f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.890240 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77af6e-a50d-4cb9-8e80-1cdc01bc4f4f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.893063 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1413/2841] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:32:51 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:32:51.894693 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b9a8f-6418-48c8-851b-6327b8e3c1a7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.895508 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971ec67-5b18-48b7-809c-94c334f35c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.896255 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971ec67-5b18-48b7-809c-94c334f35c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.896613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971ec67-5b18-48b7-809c-94c334f35c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.897303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971ec67-5b18-48b7-809c-94c334f35c7a admin admin] PATH_INFO: `/v3/projects/23336ce1c1e74b7693d45ac090b2b935/users/7f7ec5c791134db082cbb362e56e92da/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.900106 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971ec67-5b18-48b7-809c-94c334f35c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=23336ce1c1e74b7693d45ac090b2b935, user_id=7f7ec5c791134db082cbb362e56e92da, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.904604 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971ec67-5b18-48b7-809c-94c334f35c7a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.909017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b9a8f-6418-48c8-851b-6327b8e3c1a7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:51.915019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c585d00-b265-49c3-a9cd-908afd1bffd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:51.916035 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c585d00-b265-49c3-a9cd-908afd1bffd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:51.916419 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c585d00-b265-49c3-a9cd-908afd1bffd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:51.916829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c585d00-b265-49c3-a9cd-908afd1bffd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:51.917600 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c585d00-b265-49c3-a9cd-908afd1bffd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:51.919569 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c585d00-b265-49c3-a9cd-908afd1bffd4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:51.925461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1429/2842] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:51 2022] PUT /identity/v3/projects/23336ce1c1e74b7693d45ac090b2b935/users/7f7ec5c791134db082cbb362e56e92da/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:32:51.937068 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1414/2843] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:32:51.938850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971ec67-5b18-48b7-809c-94c334f35c7a admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.959079 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c585d00-b265-49c3-a9cd-908afd1bffd4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:51.961378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2c585d00-b265-49c3-a9cd-908afd1bffd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:51.977940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c585d00-b265-49c3-a9cd-908afd1bffd4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:52.048063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971ec67-5b18-48b7-809c-94c334f35c7a admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:52.057680 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2baed9-b88a-4a92-9005-aa4ef306d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:52.058299 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2baed9-b88a-4a92-9005-aa4ef306d4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:52.058723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2baed9-b88a-4a92-9005-aa4ef306d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:52.058982 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2baed9-b88a-4a92-9005-aa4ef306d4b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:52.059649 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2baed9-b88a-4a92-9005-aa4ef306d4b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:52.061298 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2baed9-b88a-4a92-9005-aa4ef306d4b1 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:52.074457 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1430/2844] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:32:52.078962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2baed9-b88a-4a92-9005-aa4ef306d4b1 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:52.086722 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:52.091298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:52.092191 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:52.092635 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:52.092761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2baed9-b88a-4a92-9005-aa4ef306d4b1 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:52.093043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] PATH_INFO: `/v3/projects/0e986dde085e45649d1e85137af83b67` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:52.094269 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e986dde085e45649d1e85137af83b67)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:52.096224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:52.099125 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1569e9-7ef4-4b84-91f1-2ac1785a7744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:52.099840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1569e9-7ef4-4b84-91f1-2ac1785a7744 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:52.100440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1569e9-7ef4-4b84-91f1-2ac1785a7744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:52.101507 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1569e9-7ef4-4b84-91f1-2ac1785a7744 admin admin] PATH_INFO: `/v3/projects/810b8a675db6466f9a26053dee6ac09d/users/84d57c43d940494198b8b28ab78c5bf8/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:52.123823 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1569e9-7ef4-4b84-91f1-2ac1785a7744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=810b8a675db6466f9a26053dee6ac09d, user_id=84d57c43d940494198b8b28ab78c5bf8, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:52.126801 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1569e9-7ef4-4b84-91f1-2ac1785a7744 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:52.151490 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1431/2845] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:32:52 2022] PUT /identity/v3/projects/810b8a675db6466f9a26053dee6ac09d/users/84d57c43d940494198b8b28ab78c5bf8/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:52.162771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26721e-be55-4d38-a1de-e23a020517c0 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:52.164036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26721e-be55-4d38-a1de-e23a020517c0 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:52.164511 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26721e-be55-4d38-a1de-e23a020517c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:52.173254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e986dde085e45649d1e85137af83b67', 'request_id': 'req-b75086b1-f202-4fcf-8989-c112d787d4ea'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:52.182366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e986dde085e45649d1e85137af83b67', 'request_id': 'req-b75086b1-f202-4fcf-8989-c112d787d4ea'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:52.191267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e986dde085e45649d1e85137af83b67', 'request_id': 'req-b75086b1-f202-4fcf-8989-c112d787d4ea'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:52.198812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9f26721e-be55-4d38-a1de-e23a020517c0 None None] MFA Rules not processed for user `84d57c43d940494198b8b28ab78c5bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:52.200280 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9f26721e-be55-4d38-a1de-e23a020517c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:52.205861 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] The token cache is being invalidate because project 0e986dde085e45649d1e85137af83b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:32:52.206288 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:52.207268 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:52.208752 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1415/2846] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:51 2022] DELETE /identity/v3/projects/0e986dde085e45649d1e85137af83b67 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:52.294674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1432/2847] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:32:52.317236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:52.319626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:52.354780 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:52.452544 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75086b1-f202-4fcf-8989-c112d787d4ea admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:52.463914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c8abd-1cb8-46f9-9ef1-c1e13582ae02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:52.465868 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c8abd-1cb8-46f9-9ef1-c1e13582ae02 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:52.466150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c8abd-1cb8-46f9-9ef1-c1e13582ae02 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:52.466867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c8abd-1cb8-46f9-9ef1-c1e13582ae02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:52.494733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c8abd-1cb8-46f9-9ef1-c1e13582ae02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:52.496520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c8abd-1cb8-46f9-9ef1-c1e13582ae02 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:52.512161 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1416/2848] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:32:56.449588 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4d156-3be5-423c-aa97-96555e614050 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:56.450346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4d156-3be5-423c-aa97-96555e614050 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:56.450346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4d156-3be5-423c-aa97-96555e614050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:56.484495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-c9b4d156-3be5-423c-aa97-96555e614050 None None] MFA Rules not processed for user `7f7ec5c791134db082cbb362e56e92da`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:56.485771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c9b4d156-3be5-423c-aa97-96555e614050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:56.590023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1433/2849] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:56.598595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c8abd-1cb8-46f9-9ef1-c1e13582ae02 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:56.600135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e26c8abd-1cb8-46f9-9ef1-c1e13582ae02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:56.661007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c8abd-1cb8-46f9-9ef1-c1e13582ae02 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:56.665425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:56.666047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:56.666311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:56.666584 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:56.668541 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:56.732439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:56.734001 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:56.743678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1417/2850] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:56.903912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d198d4-8e7f-468d-a08c-e3ea84ae83c9 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:56.904222 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d198d4-8e7f-468d-a08c-e3ea84ae83c9 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:56.904498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d198d4-8e7f-468d-a08c-e3ea84ae83c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:57.014088 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-47d198d4-8e7f-468d-a08c-e3ea84ae83c9 None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:57.015379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-47d198d4-8e7f-468d-a08c-e3ea84ae83c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:57.046366 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:57.114047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1434/2851] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 11:32:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:57.177985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:57.179685 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:57.255528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:57.289535 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705753d-b74b-40b4-a497-dd994c49fd20 service glance] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:57.295230 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a95011-c9ec-44bd-83f9-8121d58935f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:57.295958 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a95011-c9ec-44bd-83f9-8121d58935f0 service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:57.296245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a95011-c9ec-44bd-83f9-8121d58935f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:57.296525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a95011-c9ec-44bd-83f9-8121d58935f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:57.309090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a95011-c9ec-44bd-83f9-8121d58935f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:57.311134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a95011-c9ec-44bd-83f9-8121d58935f0 service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:57.321546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1418/2852] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 11:32:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 11:32:58.901602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d198d4-8e7f-468d-a08c-e3ea84ae83c9 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:58.901602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-47d198d4-8e7f-468d-a08c-e3ea84ae83c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:59.007555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d198d4-8e7f-468d-a08c-e3ea84ae83c9 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:59.012554 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:59.013440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:59.013738 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:59.014097 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] PATH_INFO: `/v3/users/84d57c43d940494198b8b28ab78c5bf8` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:59.015404 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d57c43d940494198b8b28ab78c5bf8)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:59.017646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:59.093792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d57c43d940494198b8b28ab78c5bf8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:59.094619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d57c43d940494198b8b28ab78c5bf8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:59.119012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d57c43d940494198b8b28ab78c5bf8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:59.121190 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d57c43d940494198b8b28ab78c5bf8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:59.153440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d57c43d940494198b8b28ab78c5bf8'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:32:59.186144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1435/2853] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:32:58 2022] DELETE /identity/v3/users/84d57c43d940494198b8b28ab78c5bf8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:32:59.187399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515bbb0-5d09-4d66-ad5c-22ec5e58b8ff None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:59.187912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515bbb0-5d09-4d66-ad5c-22ec5e58b8ff None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:59.188014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515bbb0-5d09-4d66-ad5c-22ec5e58b8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:59.221303 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b515bbb0-5d09-4d66-ad5c-22ec5e58b8ff None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:32:59.222526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b515bbb0-5d09-4d66-ad5c-22ec5e58b8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:59.303710 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1419/2854] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:32:59.313981 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:59.315984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:59.381176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:32:59.427124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c02f-08a7-48af-8010-caca0cc713f4 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:59.433598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d3bfc-ffcd-4672-8722-30209e8b560e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:59.434736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d3bfc-ffcd-4672-8722-30209e8b560e service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:59.435497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d3bfc-ffcd-4672-8722-30209e8b560e service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:59.435497 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d3bfc-ffcd-4672-8722-30209e8b560e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:59.449804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d3bfc-ffcd-4672-8722-30209e8b560e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:59.451915 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d3bfc-ffcd-4672-8722-30209e8b560e service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:32:59.467418 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1436/2855] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:32:59.909090 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515bbb0-5d09-4d66-ad5c-22ec5e58b8ff None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:32:59.910878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b515bbb0-5d09-4d66-ad5c-22ec5e58b8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:32:59.934796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515bbb0-5d09-4d66-ad5c-22ec5e58b8ff None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:32:59.942334 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:32:59.943202 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:32:59.943391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:32:59.943672 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] PATH_INFO: `/v3/projects/810b8a675db6466f9a26053dee6ac09d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:32:59.945430 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=810b8a675db6466f9a26053dee6ac09d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:32:59.947452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:00.031181 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810b8a675db6466f9a26053dee6ac09d', 'request_id': 'req-131c56d1-8e17-42c5-9914-44abd833e716'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:00.038663 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810b8a675db6466f9a26053dee6ac09d', 'request_id': 'req-131c56d1-8e17-42c5-9914-44abd833e716'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:00.047971 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810b8a675db6466f9a26053dee6ac09d', 'request_id': 'req-131c56d1-8e17-42c5-9914-44abd833e716'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:00.063482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] The token cache is being invalidate because project 810b8a675db6466f9a26053dee6ac09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:00.063825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:00.064564 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-131c56d1-8e17-42c5-9914-44abd833e716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:00.071769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1420/2856] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:32:59 2022] DELETE /identity/v3/projects/810b8a675db6466f9a26053dee6ac09d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:33:00.073188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908fa54-2f3a-4d07-87ef-0b381f3b01ec None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:00.073690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908fa54-2f3a-4d07-87ef-0b381f3b01ec None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:00.073778 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908fa54-2f3a-4d07-87ef-0b381f3b01ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:00.094542 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-9908fa54-2f3a-4d07-87ef-0b381f3b01ec None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:00.097989 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-9908fa54-2f3a-4d07-87ef-0b381f3b01ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:00.182350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1437/2857] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:00.184352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd458d-c6aa-45d2-9250-ff5809f41040 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:00.184648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd458d-c6aa-45d2-9250-ff5809f41040 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:00.184974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd458d-c6aa-45d2-9250-ff5809f41040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:00.208536 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-38dd458d-c6aa-45d2-9250-ff5809f41040 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:00.210645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-38dd458d-c6aa-45d2-9250-ff5809f41040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:00.219545 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1421/2858] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:00.224790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908fa54-2f3a-4d07-87ef-0b381f3b01ec None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:00.238822 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908fa54-2f3a-4d07-87ef-0b381f3b01ec None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:00.248642 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6179145-88e6-4f59-8dff-4925373b57eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:00.249445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6179145-88e6-4f59-8dff-4925373b57eb admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:00.249817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6179145-88e6-4f59-8dff-4925373b57eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:00.250183 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6179145-88e6-4f59-8dff-4925373b57eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:00.251104 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6179145-88e6-4f59-8dff-4925373b57eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:33:00.251575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6179145-88e6-4f59-8dff-4925373b57eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:00.254380 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6179145-88e6-4f59-8dff-4925373b57eb admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:00.272923 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1438/2859] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:33:00.274124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd458d-c6aa-45d2-9250-ff5809f41040 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:00.285004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd458d-c6aa-45d2-9250-ff5809f41040 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:00.290852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15c951-2d29-4c03-aacd-a0b197428a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:00.291767 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15c951-2d29-4c03-aacd-a0b197428a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:00.292408 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15c951-2d29-4c03-aacd-a0b197428a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:00.293088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15c951-2d29-4c03-aacd-a0b197428a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:00.293924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15c951-2d29-4c03-aacd-a0b197428a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:00.296112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15c951-2d29-4c03-aacd-a0b197428a92 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:00.333970 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1422/2860] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:33:00 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:00.337372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6179145-88e6-4f59-8dff-4925373b57eb admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:00.352918 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6179145-88e6-4f59-8dff-4925373b57eb admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:00.439974 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e78e35-02bf-42bf-b804-4c91879f191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:00.440741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e78e35-02bf-42bf-b804-4c91879f191e admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:00.441150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e78e35-02bf-42bf-b804-4c91879f191e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:00.441518 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e78e35-02bf-42bf-b804-4c91879f191e admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:00.442249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e78e35-02bf-42bf-b804-4c91879f191e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:00.445839 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e78e35-02bf-42bf-b804-4c91879f191e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:00.485032 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1439/2861] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:33:00 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:33:00.486578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15c951-2d29-4c03-aacd-a0b197428a92 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:00.499113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15c951-2d29-4c03-aacd-a0b197428a92 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:00.512362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da2549-286f-4a1b-bc1b-c22c8a370254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:00.513042 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da2549-286f-4a1b-bc1b-c22c8a370254 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:00.513384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da2549-286f-4a1b-bc1b-c22c8a370254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:00.513662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da2549-286f-4a1b-bc1b-c22c8a370254 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:00.514311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da2549-286f-4a1b-bc1b-c22c8a370254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:00.515957 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da2549-286f-4a1b-bc1b-c22c8a370254 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:00.533511 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1423/2862] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:33:00.536252 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e78e35-02bf-42bf-b804-4c91879f191e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:00.548297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e78e35-02bf-42bf-b804-4c91879f191e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:00.554303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa7410-9dea-4cef-9ffb-0ce3732ce98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:00.556259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa7410-9dea-4cef-9ffb-0ce3732ce98d admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:00.556259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa7410-9dea-4cef-9ffb-0ce3732ce98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:00.556259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa7410-9dea-4cef-9ffb-0ce3732ce98d admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:00.556764 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa7410-9dea-4cef-9ffb-0ce3732ce98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:00.559803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa7410-9dea-4cef-9ffb-0ce3732ce98d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:00.576849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1440/2863] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:00.579065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da2549-286f-4a1b-bc1b-c22c8a370254 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:00.590473 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da2549-286f-4a1b-bc1b-c22c8a370254 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:00.595329 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a35d1-d28e-4baa-849d-66efc4720afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:00.595993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a35d1-d28e-4baa-849d-66efc4720afb admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:00.596273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a35d1-d28e-4baa-849d-66efc4720afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:00.596546 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a35d1-d28e-4baa-849d-66efc4720afb admin admin] PATH_INFO: `/v3/projects/96b2aa79464a4c50962e3f8bd45ded14/users/622d64d818c84f5ba7a84816ec35e516/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:00.605435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:00.614765 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a35d1-d28e-4baa-849d-66efc4720afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b2aa79464a4c50962e3f8bd45ded14, user_id=622d64d818c84f5ba7a84816ec35e516, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:00.618737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a35d1-d28e-4baa-849d-66efc4720afb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:00.651526 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1424/2864] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:00 2022] PUT /identity/v3/projects/96b2aa79464a4c50962e3f8bd45ded14/users/622d64d818c84f5ba7a84816ec35e516/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:00.653827 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292faab-a4f1-40f7-bb7e-a4c1f20c559d None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:00.654378 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292faab-a4f1-40f7-bb7e-a4c1f20c559d None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:00.654866 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292faab-a4f1-40f7-bb7e-a4c1f20c559d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:00.691717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-0292faab-a4f1-40f7-bb7e-a4c1f20c559d None None] MFA Rules not processed for user `622d64d818c84f5ba7a84816ec35e516`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:00.693083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-0292faab-a4f1-40f7-bb7e-a4c1f20c559d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:00.750350 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:00.821884 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1441/2865] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:00.844093 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a35d1-d28e-4baa-849d-66efc4720afb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:00.846429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c78a35d1-d28e-4baa-849d-66efc4720afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:01.033347 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a35d1-d28e-4baa-849d-66efc4720afb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:01.045585 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10529164-ce8b-40f0-9e5b-73f58710e629 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:01.046320 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10529164-ce8b-40f0-9e5b-73f58710e629 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:01.046729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10529164-ce8b-40f0-9e5b-73f58710e629 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:01.047078 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10529164-ce8b-40f0-9e5b-73f58710e629 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:01.058182 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10529164-ce8b-40f0-9e5b-73f58710e629 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:01.059897 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10529164-ce8b-40f0-9e5b-73f58710e629 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:01.075254 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1425/2866] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:01.384267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292faab-a4f1-40f7-bb7e-a4c1f20c559d None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:01.513798 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292faab-a4f1-40f7-bb7e-a4c1f20c559d None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:01.529576 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:01.530389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:01.530920 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:01.533890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] PATH_INFO: `/v3/users/622d64d818c84f5ba7a84816ec35e516` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:01.535826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=622d64d818c84f5ba7a84816ec35e516)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:01.539370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:01.623586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '622d64d818c84f5ba7a84816ec35e516'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:01.624555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '622d64d818c84f5ba7a84816ec35e516'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:01.635525 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '622d64d818c84f5ba7a84816ec35e516'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:01.636295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '622d64d818c84f5ba7a84816ec35e516'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:01.645887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '622d64d818c84f5ba7a84816ec35e516'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:01.678071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1442/2867] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:33:01 2022] DELETE /identity/v3/users/622d64d818c84f5ba7a84816ec35e516 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:01.680300 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1323b-9919-461e-a202-f1e18d758964 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:01.681924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1323b-9919-461e-a202-f1e18d758964 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:01.682218 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1323b-9919-461e-a202-f1e18d758964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:01.709474 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3aa1323b-9919-461e-a202-f1e18d758964 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:01.710903 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3aa1323b-9919-461e-a202-f1e18d758964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:01.801680 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1426/2868] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:01.812485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:01.814009 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:01.910563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bf8d2-a8d4-494b-be30-e2d434771a1c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:01.922721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027be75-7406-4061-884e-e8fd286ce17a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:01.923358 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027be75-7406-4061-884e-e8fd286ce17a service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:01.923638 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027be75-7406-4061-884e-e8fd286ce17a service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:01.923912 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027be75-7406-4061-884e-e8fd286ce17a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:01.934188 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:01.937437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027be75-7406-4061-884e-e8fd286ce17a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:01.939140 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027be75-7406-4061-884e-e8fd286ce17a service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:01.952123 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1443/2869] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:02.329206 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1323b-9919-461e-a202-f1e18d758964 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:02.330990 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3aa1323b-9919-461e-a202-f1e18d758964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:02.366232 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1323b-9919-461e-a202-f1e18d758964 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:02.371821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:02.372559 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:02.372963 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:02.373310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] PATH_INFO: `/v3/projects/96b2aa79464a4c50962e3f8bd45ded14` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:02.374503 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b2aa79464a4c50962e3f8bd45ded14)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:02.376940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:02.435794 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b2aa79464a4c50962e3f8bd45ded14', 'request_id': 'req-559b7555-c41d-4ae4-978b-d9312405ebf3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:02.441433 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b2aa79464a4c50962e3f8bd45ded14', 'request_id': 'req-559b7555-c41d-4ae4-978b-d9312405ebf3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:02.451660 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b2aa79464a4c50962e3f8bd45ded14', 'request_id': 'req-559b7555-c41d-4ae4-978b-d9312405ebf3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:02.453870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:02.470245 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] The token cache is being invalidate because project 96b2aa79464a4c50962e3f8bd45ded14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:02.470950 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:02.471981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-559b7555-c41d-4ae4-978b-d9312405ebf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:02.473812 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1427/2870] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:33:02 2022] DELETE /identity/v3/projects/96b2aa79464a4c50962e3f8bd45ded14 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:02.478887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e50525-1885-49ab-9161-a2f802b7a356 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:02.479368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e50525-1885-49ab-9161-a2f802b7a356 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:02.479450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e50525-1885-49ab-9161-a2f802b7a356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:02.509452 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-b5e50525-1885-49ab-9161-a2f802b7a356 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:02.510833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b5e50525-1885-49ab-9161-a2f802b7a356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:02.591731 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1444/2871] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:02.601312 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d14e9-0b7b-4d10-84c7-ea14c5a30779 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:02.601598 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d14e9-0b7b-4d10-84c7-ea14c5a30779 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:02.601878 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d14e9-0b7b-4d10-84c7-ea14c5a30779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:02.625505 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-568d14e9-0b7b-4d10-84c7-ea14c5a30779 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:02.626796 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-568d14e9-0b7b-4d10-84c7-ea14c5a30779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:02.641262 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1428/2872] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:02.642437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e50525-1885-49ab-9161-a2f802b7a356 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:02.654699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e50525-1885-49ab-9161-a2f802b7a356 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:02.659958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44b569-cea4-4e79-a6c9-cb427fb1ecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:02.660563 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44b569-cea4-4e79-a6c9-cb427fb1ecc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:02.660869 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44b569-cea4-4e79-a6c9-cb427fb1ecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:02.661141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44b569-cea4-4e79-a6c9-cb427fb1ecc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:02.661780 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44b569-cea4-4e79-a6c9-cb427fb1ecc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:33:02.662078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44b569-cea4-4e79-a6c9-cb427fb1ecc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:02.664044 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44b569-cea4-4e79-a6c9-cb427fb1ecc2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:02.679800 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1445/2873] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:33:02.684391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d14e9-0b7b-4d10-84c7-ea14c5a30779 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:02.695273 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d14e9-0b7b-4d10-84c7-ea14c5a30779 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:02.700337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc7f8-789c-4b3d-87a9-b0b5609ace6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:02.701047 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc7f8-789c-4b3d-87a9-b0b5609ace6b admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:02.701394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc7f8-789c-4b3d-87a9-b0b5609ace6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:02.701742 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc7f8-789c-4b3d-87a9-b0b5609ace6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:02.702435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afc7f8-789c-4b3d-87a9-b0b5609ace6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:02.704403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afc7f8-789c-4b3d-87a9-b0b5609ace6b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:02.744743 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1429/2874] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:33:02 2022] POST /identity/v3/projects => generated 399 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:02.750464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44b569-cea4-4e79-a6c9-cb427fb1ecc2 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:02.762214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44b569-cea4-4e79-a6c9-cb427fb1ecc2 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:02.856979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74854c-14ec-4908-aa59-3e309b2a8a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:02.857597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74854c-14ec-4908-aa59-3e309b2a8a9a admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:02.857877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74854c-14ec-4908-aa59-3e309b2a8a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:02.858363 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74854c-14ec-4908-aa59-3e309b2a8a9a admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:02.858948 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74854c-14ec-4908-aa59-3e309b2a8a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:02.861036 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74854c-14ec-4908-aa59-3e309b2a8a9a admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:02.892825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1446/2875] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:33:02 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:02.893885 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc7f8-789c-4b3d-87a9-b0b5609ace6b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:02.919589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc7f8-789c-4b3d-87a9-b0b5609ace6b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:02.933578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cfaa4-5571-4ad6-9317-32750e932b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:02.934301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cfaa4-5571-4ad6-9317-32750e932b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:02.934658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cfaa4-5571-4ad6-9317-32750e932b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:02.935004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cfaa4-5571-4ad6-9317-32750e932b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:02.935735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cfaa4-5571-4ad6-9317-32750e932b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:02.937492 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cfaa4-5571-4ad6-9317-32750e932b20 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:02.949248 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1430/2876] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:02.953505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74854c-14ec-4908-aa59-3e309b2a8a9a admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:02.965665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74854c-14ec-4908-aa59-3e309b2a8a9a admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:02.971498 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1697d-2dae-4168-82b1-1fe58b63d039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:02.972277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1697d-2dae-4168-82b1-1fe58b63d039 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:02.972651 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1697d-2dae-4168-82b1-1fe58b63d039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:02.973023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1697d-2dae-4168-82b1-1fe58b63d039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:02.973747 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1697d-2dae-4168-82b1-1fe58b63d039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:02.977226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1697d-2dae-4168-82b1-1fe58b63d039 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:02.988023 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1447/2877] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 11:33:02.992043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cfaa4-5571-4ad6-9317-32750e932b20 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:03.002924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cfaa4-5571-4ad6-9317-32750e932b20 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:03.008968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38528d0-16b0-46b2-afb2-62948dcd91f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:03.009733 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38528d0-16b0-46b2-afb2-62948dcd91f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.010088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38528d0-16b0-46b2-afb2-62948dcd91f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.010440 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38528d0-16b0-46b2-afb2-62948dcd91f4 admin admin] PATH_INFO: `/v3/projects/5c7ac4eb70f0462d98ef9e3a2843738b/users/43c38f392c244d4a80018b75e0d2555a/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.029898 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38528d0-16b0-46b2-afb2-62948dcd91f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7ac4eb70f0462d98ef9e3a2843738b, user_id=43c38f392c244d4a80018b75e0d2555a, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:03.033708 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38528d0-16b0-46b2-afb2-62948dcd91f4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:03.053199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1431/2878] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:02 2022] PUT /identity/v3/projects/5c7ac4eb70f0462d98ef9e3a2843738b/users/43c38f392c244d4a80018b75e0d2555a/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:33:03.063535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f21259-70b3-4ed8-b440-6ea6de1d3662 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.063828 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f21259-70b3-4ed8-b440-6ea6de1d3662 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.064103 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f21259-70b3-4ed8-b440-6ea6de1d3662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.101850 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-18f21259-70b3-4ed8-b440-6ea6de1d3662 None None] MFA Rules not processed for user `43c38f392c244d4a80018b75e0d2555a`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:03.103274 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-18f21259-70b3-4ed8-b440-6ea6de1d3662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:03.166249 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:03.187613 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1448/2879] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:03.192199 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38528d0-16b0-46b2-afb2-62948dcd91f4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:03.264014 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38528d0-16b0-46b2-afb2-62948dcd91f4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:03.268124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02a161-ba58-468e-a662-00949e999a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:03.268749 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02a161-ba58-468e-a662-00949e999a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.269054 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02a161-ba58-468e-a662-00949e999a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.269336 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02a161-ba58-468e-a662-00949e999a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.269968 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02a161-ba58-468e-a662-00949e999a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:03.271943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02a161-ba58-468e-a662-00949e999a45 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:03.301553 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1432/2880] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:33:03 2022] POST /identity/v3/projects => generated 399 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:03.306144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f21259-70b3-4ed8-b440-6ea6de1d3662 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:03.318155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f21259-70b3-4ed8-b440-6ea6de1d3662 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:03.384760 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fe7a4-2832-4c05-b39c-806b52723f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:03.385467 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fe7a4-2832-4c05-b39c-806b52723f86 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.385757 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fe7a4-2832-4c05-b39c-806b52723f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.386030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fe7a4-2832-4c05-b39c-806b52723f86 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.386833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fe7a4-2832-4c05-b39c-806b52723f86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:03.388820 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fe7a4-2832-4c05-b39c-806b52723f86 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:03.415987 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1449/2881] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:33:03 2022] POST /identity/v3/users => generated 306 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:03.419834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02a161-ba58-468e-a662-00949e999a45 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:03.432010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02a161-ba58-468e-a662-00949e999a45 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:03.437856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03f02e-5cef-4a96-b8fe-00b2477ac5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:03.438579 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03f02e-5cef-4a96-b8fe-00b2477ac5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.438959 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03f02e-5cef-4a96-b8fe-00b2477ac5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.439301 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03f02e-5cef-4a96-b8fe-00b2477ac5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.439956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03f02e-5cef-4a96-b8fe-00b2477ac5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:03.441814 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03f02e-5cef-4a96-b8fe-00b2477ac5c9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:03.455223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1433/2882] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:03.458598 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fe7a4-2832-4c05-b39c-806b52723f86 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:03.469871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fe7a4-2832-4c05-b39c-806b52723f86 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:03.477456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0ddf8-4fb3-479e-bb02-84223feee4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:03.478131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0ddf8-4fb3-479e-bb02-84223feee4fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.478408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0ddf8-4fb3-479e-bb02-84223feee4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.478688 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0ddf8-4fb3-479e-bb02-84223feee4fa admin admin] PATH_INFO: `/v3/projects/87d2c84a451248d2ae5d81aceaadbe23/users/de86765fb06740bb95c93a56e688f69b/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.499953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0ddf8-4fb3-479e-bb02-84223feee4fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d2c84a451248d2ae5d81aceaadbe23, user_id=de86765fb06740bb95c93a56e688f69b, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:03.502709 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0ddf8-4fb3-479e-bb02-84223feee4fa admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:03.519294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1450/2883] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:03 2022] PUT /identity/v3/projects/87d2c84a451248d2ae5d81aceaadbe23/users/de86765fb06740bb95c93a56e688f69b/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:03.523400 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03f02e-5cef-4a96-b8fe-00b2477ac5c9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:03.535941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03f02e-5cef-4a96-b8fe-00b2477ac5c9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:03.622690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ae326-2f08-4d56-9229-7f06edebf12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:03.623405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ae326-2f08-4d56-9229-7f06edebf12a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.623692 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ae326-2f08-4d56-9229-7f06edebf12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.623992 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ae326-2f08-4d56-9229-7f06edebf12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.624631 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ae326-2f08-4d56-9229-7f06edebf12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:03.626323 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ae326-2f08-4d56-9229-7f06edebf12a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:03.642178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1434/2884] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:03.646351 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0ddf8-4fb3-479e-bb02-84223feee4fa admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:03.658463 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0ddf8-4fb3-479e-bb02-84223feee4fa admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:03.664509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b492b-ce37-49b3-828c-ab359f726df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:03.665456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b492b-ce37-49b3-828c-ab359f726df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.666048 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b492b-ce37-49b3-828c-ab359f726df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.666474 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b492b-ce37-49b3-828c-ab359f726df8 admin admin] PATH_INFO: `/v3/projects/87d2c84a451248d2ae5d81aceaadbe23/users/de86765fb06740bb95c93a56e688f69b/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.669131 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b492b-ce37-49b3-828c-ab359f726df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d2c84a451248d2ae5d81aceaadbe23, user_id=de86765fb06740bb95c93a56e688f69b, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:03.671840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b492b-ce37-49b3-828c-ab359f726df8 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:03.688241 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1451/2885] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:03 2022] PUT /identity/v3/projects/87d2c84a451248d2ae5d81aceaadbe23/users/de86765fb06740bb95c93a56e688f69b/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:03.702695 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc183a71-79ca-4657-b483-afff22355a24 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.703227 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc183a71-79ca-4657-b483-afff22355a24 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.703357 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc183a71-79ca-4657-b483-afff22355a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.712355 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:03.736791 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dc183a71-79ca-4657-b483-afff22355a24 None None] MFA Rules not processed for user `de86765fb06740bb95c93a56e688f69b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:03.738031 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dc183a71-79ca-4657-b483-afff22355a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:03.828284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1435/2886] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:03.840833 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b492b-ce37-49b3-828c-ab359f726df8 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:03.842632 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-396b492b-ce37-49b3-828c-ab359f726df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:03.919541 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b492b-ce37-49b3-828c-ab359f726df8 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:03.923293 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e80cb-7c67-4ac9-a4be-3201b5fdc040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:03.923907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e80cb-7c67-4ac9-a4be-3201b5fdc040 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:03.924189 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e80cb-7c67-4ac9-a4be-3201b5fdc040 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:03.924462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e80cb-7c67-4ac9-a4be-3201b5fdc040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:03.935674 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e80cb-7c67-4ac9-a4be-3201b5fdc040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:03.937174 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e80cb-7c67-4ac9-a4be-3201b5fdc040 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:03.947988 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1452/2887] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:33:04.050127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc183a71-79ca-4657-b483-afff22355a24 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:04.063007 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc183a71-79ca-4657-b483-afff22355a24 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:04.077912 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ea4c5-f719-40ce-acca-e9422e509da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:04.078609 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ea4c5-f719-40ce-acca-e9422e509da4 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:04.078837 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ea4c5-f719-40ce-acca-e9422e509da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:04.079140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ea4c5-f719-40ce-acca-e9422e509da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:04.156102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ea4c5-f719-40ce-acca-e9422e509da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:04.157622 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ea4c5-f719-40ce-acca-e9422e509da4 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:04.171228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1436/2888] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:04 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:07.632030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee0bfa-f0c7-4401-ad74-93a335b84570 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:07.632571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee0bfa-f0c7-4401-ad74-93a335b84570 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:07.632720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee0bfa-f0c7-4401-ad74-93a335b84570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:07.652624 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e9ee0bfa-f0c7-4401-ad74-93a335b84570 None None] MFA Rules not processed for user `53d8bbb16c794d13bf005e21b832c87e`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:07.655038 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e9ee0bfa-f0c7-4401-ad74-93a335b84570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:07.701505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:07.717552 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1453/2889] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:07.980088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ea4c5-f719-40ce-acca-e9422e509da4 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:07.981555 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-810ea4c5-f719-40ce-acca-e9422e509da4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:08.037785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ea4c5-f719-40ce-acca-e9422e509da4 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:08.041506 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c9672-dcff-4e7b-ae43-0bff68ee71d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:08.042112 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c9672-dcff-4e7b-ae43-0bff68ee71d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:08.042391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c9672-dcff-4e7b-ae43-0bff68ee71d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:08.042662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c9672-dcff-4e7b-ae43-0bff68ee71d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:08.044378 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-902c9672-dcff-4e7b-ae43-0bff68ee71d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:08.054642 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c9672-dcff-4e7b-ae43-0bff68ee71d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:08.056340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c9672-dcff-4e7b-ae43-0bff68ee71d6 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:08.064534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1437/2890] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:08.399269 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee0bfa-f0c7-4401-ad74-93a335b84570 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:08.401753 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e9ee0bfa-f0c7-4401-ad74-93a335b84570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:08.416581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee0bfa-f0c7-4401-ad74-93a335b84570 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:08.420579 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806f492-6ee6-4857-9a52-8071cfe8417c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:08.421395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806f492-6ee6-4857-9a52-8071cfe8417c service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:08.421840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806f492-6ee6-4857-9a52-8071cfe8417c service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:08.422265 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806f492-6ee6-4857-9a52-8071cfe8417c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:08.424155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6806f492-6ee6-4857-9a52-8071cfe8417c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:08.470291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c9672-dcff-4e7b-ae43-0bff68ee71d6 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:08.508140 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:08.525765 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806f492-6ee6-4857-9a52-8071cfe8417c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:08.527210 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806f492-6ee6-4857-9a52-8071cfe8417c service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:08.540389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1454/2891] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:33:08.558130 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c9672-dcff-4e7b-ae43-0bff68ee71d6 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:08.563800 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:08.564612 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:08.565432 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:08.565892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] PATH_INFO: `/v3/users/43c38f392c244d4a80018b75e0d2555a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:08.567188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] RBAC: Authorizing `identity:delete_user(user_id=43c38f392c244d4a80018b75e0d2555a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:08.570234 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:08.636385 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43c38f392c244d4a80018b75e0d2555a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.645918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43c38f392c244d4a80018b75e0d2555a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.648038 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43c38f392c244d4a80018b75e0d2555a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.656595 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43c38f392c244d4a80018b75e0d2555a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.657330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43c38f392c244d4a80018b75e0d2555a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.683825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1438/2892] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:33:08 2022] DELETE /identity/v3/users/43c38f392c244d4a80018b75e0d2555a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:33:08.688005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127ebe6-fd74-4ca3-8efc-e8d9bbaebe35 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:08.688440 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127ebe6-fd74-4ca3-8efc-e8d9bbaebe35 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:08.688546 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127ebe6-fd74-4ca3-8efc-e8d9bbaebe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:08.714913 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-d127ebe6-fd74-4ca3-8efc-e8d9bbaebe35 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:08.716382 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d127ebe6-fd74-4ca3-8efc-e8d9bbaebe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:08.724487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:08.726468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:08.794714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1455/2893] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:08.801776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127ebe6-fd74-4ca3-8efc-e8d9bbaebe35 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:08.803792 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d127ebe6-fd74-4ca3-8efc-e8d9bbaebe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:08.804143 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537608e0-9617-496b-bea7-9b90ae27464e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:08.809939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:08.811187 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:08.811636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:08.811919 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] PATH_INFO: `/v3/users/964301bb43684a25b0cc10855faf6cae` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:08.813124 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] RBAC: Authorizing `identity:delete_user(user_id=964301bb43684a25b0cc10855faf6cae)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:08.815105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:08.871593 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '964301bb43684a25b0cc10855faf6cae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.876002 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:08.878079 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127ebe6-fd74-4ca3-8efc-e8d9bbaebe35 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:08.879592 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '964301bb43684a25b0cc10855faf6cae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.880178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '964301bb43684a25b0cc10855faf6cae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.882569 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99097329-1f7a-4261-9dda-d7ddb61352d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:08.883995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99097329-1f7a-4261-9dda-d7ddb61352d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:08.884267 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99097329-1f7a-4261-9dda-d7ddb61352d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:08.884485 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99097329-1f7a-4261-9dda-d7ddb61352d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:08.890455 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '964301bb43684a25b0cc10855faf6cae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.891369 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '964301bb43684a25b0cc10855faf6cae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:08.896874 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99097329-1f7a-4261-9dda-d7ddb61352d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:08.898509 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99097329-1f7a-4261-9dda-d7ddb61352d4 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:08.908629 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1456/2894] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:33:08.913017 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1439/2895] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:33:08 2022] DELETE /identity/v3/users/964301bb43684a25b0cc10855faf6cae => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:33:08.917175 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8a197-1a62-4c1a-88d7-fd1db1a9b4fd None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:08.917549 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8a197-1a62-4c1a-88d7-fd1db1a9b4fd None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:08.917856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8a197-1a62-4c1a-88d7-fd1db1a9b4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:08.941297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-83e8a197-1a62-4c1a-88d7-fd1db1a9b4fd None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:08.942887 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-83e8a197-1a62-4c1a-88d7-fd1db1a9b4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:08.948962 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=100 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:09.017283 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1457/2896] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:33:09.039551 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:09.042224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:09.123964 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744bffc-4d61-476d-9236-425ab794814e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:09.128406 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc0dd8-3ef7-481a-9d59-e7ee066a7d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:09.129418 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc0dd8-3ef7-481a-9d59-e7ee066a7d21 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:09.129706 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc0dd8-3ef7-481a-9d59-e7ee066a7d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:09.129983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc0dd8-3ef7-481a-9d59-e7ee066a7d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:09.156623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc0dd8-3ef7-481a-9d59-e7ee066a7d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:09.158207 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc0dd8-3ef7-481a-9d59-e7ee066a7d21 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:09.169851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1440/2897] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:33:09.268654 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8a197-1a62-4c1a-88d7-fd1db1a9b4fd None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:09.270186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-83e8a197-1a62-4c1a-88d7-fd1db1a9b4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:09.283721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8a197-1a62-4c1a-88d7-fd1db1a9b4fd None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:09.293790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:09.294503 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:09.294804 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:09.295199 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] PATH_INFO: `/v3/projects/5c7ac4eb70f0462d98ef9e3a2843738b` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:09.296434 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c7ac4eb70f0462d98ef9e3a2843738b)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:09.299020 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:09.350909 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c7ac4eb70f0462d98ef9e3a2843738b', 'request_id': 'req-d439582c-b087-4dd4-8b31-2f7fbaff57f7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.359321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c7ac4eb70f0462d98ef9e3a2843738b', 'request_id': 'req-d439582c-b087-4dd4-8b31-2f7fbaff57f7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.369665 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c7ac4eb70f0462d98ef9e3a2843738b', 'request_id': 'req-d439582c-b087-4dd4-8b31-2f7fbaff57f7'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.382429 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] The token cache is being invalidate because project 5c7ac4eb70f0462d98ef9e3a2843738b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:09.382720 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.383396 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.384949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1458/2898] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:33:09 2022] DELETE /identity/v3/projects/5c7ac4eb70f0462d98ef9e3a2843738b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:09.389241 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc0dd8-3ef7-481a-9d59-e7ee066a7d21 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:09.401666 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc0dd8-3ef7-481a-9d59-e7ee066a7d21 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:09.473884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:09.474662 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:09.475135 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:09.475256 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] PATH_INFO: `/v3/users/de86765fb06740bb95c93a56e688f69b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:09.476554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=de86765fb06740bb95c93a56e688f69b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:09.479084 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:09.507389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:09.509122 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:09.534152 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439582c-b087-4dd4-8b31-2f7fbaff57f7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:09.552146 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:09.554615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:09.554968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:09.555602 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] PATH_INFO: `/v3/projects/45a9d11f80d64ea2b9ba6a32b06320e4` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:09.567462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de86765fb06740bb95c93a56e688f69b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.571581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a9d11f80d64ea2b9ba6a32b06320e4)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:09.572840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:09.578362 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de86765fb06740bb95c93a56e688f69b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.579279 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de86765fb06740bb95c93a56e688f69b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.590013 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de86765fb06740bb95c93a56e688f69b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.591100 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c2675f89-9321-4dec-9bbb-cb76a0dfb05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de86765fb06740bb95c93a56e688f69b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.600215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:09.624203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1441/2899] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:33:09 2022] DELETE /identity/v3/users/de86765fb06740bb95c93a56e688f69b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:09.680282 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a9d11f80d64ea2b9ba6a32b06320e4', 'request_id': 'req-60f21391-eb42-40be-86ff-16d419079625'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.695868 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a9d11f80d64ea2b9ba6a32b06320e4', 'request_id': 'req-60f21391-eb42-40be-86ff-16d419079625'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.708153 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a9d11f80d64ea2b9ba6a32b06320e4', 'request_id': 'req-60f21391-eb42-40be-86ff-16d419079625'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.724480 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] The token cache is being invalidate because project 45a9d11f80d64ea2b9ba6a32b06320e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:09.725352 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.726143 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-60f21391-eb42-40be-86ff-16d419079625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:09.729291 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1459/2900] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:33:09 2022] DELETE /identity/v3/projects/45a9d11f80d64ea2b9ba6a32b06320e4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:33:09.732941 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88a08a-bb72-4634-9cb6-8b22a54a1e26 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:09.733487 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88a08a-bb72-4634-9cb6-8b22a54a1e26 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:09.733983 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88a08a-bb72-4634-9cb6-8b22a54a1e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:09.759633 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0e88a08a-bb72-4634-9cb6-8b22a54a1e26 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:09.760985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0e88a08a-bb72-4634-9cb6-8b22a54a1e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:09.837884 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1442/2901] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:09.844949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31746b1c-e242-43e5-a317-910832ba4c1a None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:09.845238 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31746b1c-e242-43e5-a317-910832ba4c1a None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:09.845516 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31746b1c-e242-43e5-a317-910832ba4c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:09.867258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-31746b1c-e242-43e5-a317-910832ba4c1a None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:09.868581 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-31746b1c-e242-43e5-a317-910832ba4c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:09.877289 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56896190-4d6b-4918-a6ee-9a23f7345ea1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:09.877647 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56896190-4d6b-4918-a6ee-9a23f7345ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:09.878074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56896190-4d6b-4918-a6ee-9a23f7345ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:09.879583 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1460/2902] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:09.884464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31746b1c-e242-43e5-a317-910832ba4c1a None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:09.893675 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31746b1c-e242-43e5-a317-910832ba4c1a None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:09.898846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0c87d-d905-44d1-b11c-2880990d1a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:09.899461 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0c87d-d905-44d1-b11c-2880990d1a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:09.899743 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0c87d-d905-44d1-b11c-2880990d1a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:09.900017 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0c87d-d905-44d1-b11c-2880990d1a60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:09.900668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0c87d-d905-44d1-b11c-2880990d1a60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:33:09.900999 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0c87d-d905-44d1-b11c-2880990d1a60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:09.902737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0c87d-d905-44d1-b11c-2880990d1a60 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:09.905444 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:09.908462 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-56896190-4d6b-4918-a6ee-9a23f7345ea1 None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:09.910136 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-56896190-4d6b-4918-a6ee-9a23f7345ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:09.921341 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1461/2903] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 11:33:09.922331 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0c87d-d905-44d1-b11c-2880990d1a60 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:09.935217 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0c87d-d905-44d1-b11c-2880990d1a60 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:09.941739 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467fe939-9937-451e-a1a5-18c49d4f12de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:09.942355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fe939-9937-451e-a1a5-18c49d4f12de admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:09.942646 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fe939-9937-451e-a1a5-18c49d4f12de admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:09.942917 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fe939-9937-451e-a1a5-18c49d4f12de admin admin] PATH_INFO: `/v3/projects` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:09.943558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467fe939-9937-451e-a1a5-18c49d4f12de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:09.945812 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467fe939-9937-451e-a1a5-18c49d4f12de admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:09.977547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1462/2904] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:33:09 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:09.981547 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467fe939-9937-451e-a1a5-18c49d4f12de admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:09.986578 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1443/2905] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Sep 15 11:33:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:33:10.000449 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56896190-4d6b-4918-a6ee-9a23f7345ea1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:10.002533 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-56896190-4d6b-4918-a6ee-9a23f7345ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:10.063355 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467fe939-9937-451e-a1a5-18c49d4f12de admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:10.068641 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd7477-2801-46ab-b639-d68b0b7ff494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:10.069310 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd7477-2801-46ab-b639-d68b0b7ff494 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.069648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd7477-2801-46ab-b639-d68b0b7ff494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.069856 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd7477-2801-46ab-b639-d68b0b7ff494 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:10.070574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd7477-2801-46ab-b639-d68b0b7ff494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:10.072450 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd7477-2801-46ab-b639-d68b0b7ff494 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:10.090871 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56896190-4d6b-4918-a6ee-9a23f7345ea1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:10.101083 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1463/2906] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:33:09 2022] POST /identity/v3/users => generated 285 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:10.105694 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd7477-2801-46ab-b639-d68b0b7ff494 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:10.117277 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd7477-2801-46ab-b639-d68b0b7ff494 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:10.122112 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e56b74-7249-4d27-a7da-f78df501c251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:10.122736 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e56b74-7249-4d27-a7da-f78df501c251 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.123016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e56b74-7249-4d27-a7da-f78df501c251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.123294 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e56b74-7249-4d27-a7da-f78df501c251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:10.123941 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e56b74-7249-4d27-a7da-f78df501c251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:10.125617 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e56b74-7249-4d27-a7da-f78df501c251 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:10.137532 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1464/2907] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:10.138294 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:10.141889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e56b74-7249-4d27-a7da-f78df501c251 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:10.153206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e56b74-7249-4d27-a7da-f78df501c251 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:10.158459 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae3d35-dcac-44ec-be1e-7247f275b61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:10.159409 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae3d35-dcac-44ec-be1e-7247f275b61d admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.159977 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae3d35-dcac-44ec-be1e-7247f275b61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.160254 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae3d35-dcac-44ec-be1e-7247f275b61d admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:10.160955 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae3d35-dcac-44ec-be1e-7247f275b61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:10.162871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae3d35-dcac-44ec-be1e-7247f275b61d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:10.168844 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc4de4-6d8c-4fbc-acf3-ecac14872095 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:10.169472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc4de4-6d8c-4fbc-acf3-ecac14872095 service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.169760 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc4de4-6d8c-4fbc-acf3-ecac14872095 service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.170032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc4de4-6d8c-4fbc-acf3-ecac14872095 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:10.174726 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1465/2908] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:10.179417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae3d35-dcac-44ec-be1e-7247f275b61d admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:10.191871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae3d35-dcac-44ec-be1e-7247f275b61d admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:10.197619 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2921e-6a0b-471e-af98-d2bcbd474fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:10.198353 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2921e-6a0b-471e-af98-d2bcbd474fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.198782 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2921e-6a0b-471e-af98-d2bcbd474fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.199063 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2921e-6a0b-471e-af98-d2bcbd474fc2 admin admin] PATH_INFO: `/v3/projects/3aebb788805e42deb30b647dd53e4c62/users/3bfc300aefab4a22a949d4807073d5d1/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:10.216466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2921e-6a0b-471e-af98-d2bcbd474fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aebb788805e42deb30b647dd53e4c62, user_id=3bfc300aefab4a22a949d4807073d5d1, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:10.219197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2921e-6a0b-471e-af98-d2bcbd474fc2 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:10.234659 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1466/2909] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:10 2022] PUT /identity/v3/projects/3aebb788805e42deb30b647dd53e4c62/users/3bfc300aefab4a22a949d4807073d5d1/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:33:10.235081 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc4de4-6d8c-4fbc-acf3-ecac14872095 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:10.236678 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc4de4-6d8c-4fbc-acf3-ecac14872095 service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:10.245540 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604d6fb-471e-406f-b828-b2463c587e86 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.245826 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604d6fb-471e-406f-b828-b2463c587e86 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.246100 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604d6fb-471e-406f-b828-b2463c587e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:10.250270 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1444/2910] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 11:33:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 11:33:10.277741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-3604d6fb-471e-406f-b828-b2463c587e86 None None] MFA Rules not processed for user `3bfc300aefab4a22a949d4807073d5d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:10.278997 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3604d6fb-471e-406f-b828-b2463c587e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:10.360010 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc4de4-6d8c-4fbc-acf3-ecac14872095 service swift] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:10.361778 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-49dc4de4-6d8c-4fbc-acf3-ecac14872095 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:10.385539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc4de4-6d8c-4fbc-acf3-ecac14872095 service swift] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:10.438537 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1467/2911] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:33:10.451411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604d6fb-471e-406f-b828-b2463c587e86 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:10.453776 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-3604d6fb-471e-406f-b828-b2463c587e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:10.478771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:10.549155 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:10.551166 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.551773 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.558965 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] PATH_INFO: `/v3/projects/87d2c84a451248d2ae5d81aceaadbe23` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:10.561443 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d2c84a451248d2ae5d81aceaadbe23)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:10.563621 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:10.583043 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604d6fb-471e-406f-b828-b2463c587e86 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:10.602081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7e3ca-f897-4667-b8aa-ceeb2e2e65f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6908a326a3d3437c9db898976a2def1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:10.603212 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7e3ca-f897-4667-b8aa-ceeb2e2e65f0 service glance] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.603715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7e3ca-f897-4667-b8aa-ceeb2e2e65f0 service glance] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.604206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7e3ca-f897-4667-b8aa-ceeb2e2e65f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:10.626668 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7e3ca-f897-4667-b8aa-ceeb2e2e65f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:10.629047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7e3ca-f897-4667-b8aa-ceeb2e2e65f0 service glance] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:10.651395 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1468/2912] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:10.666699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d2c84a451248d2ae5d81aceaadbe23', 'request_id': 'req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:10.675274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d2c84a451248d2ae5d81aceaadbe23', 'request_id': 'req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:10.686849 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d2c84a451248d2ae5d81aceaadbe23', 'request_id': 'req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:10.703956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] The token cache is being invalidate because project 87d2c84a451248d2ae5d81aceaadbe23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:10.716311 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:10.718565 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0cb06fa0-ecbe-4ad3-a1d5-c4ebae60cfc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:10.724614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1445/2913] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:33:10 2022] DELETE /identity/v3/projects/87d2c84a451248d2ae5d81aceaadbe23 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:10.730170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e1be4-a9dc-46f7-b3d0-754bbf0cb068 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.731224 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e1be4-a9dc-46f7-b3d0-754bbf0cb068 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.732158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e1be4-a9dc-46f7-b3d0-754bbf0cb068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:10.780664 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-194e1be4-a9dc-46f7-b3d0-754bbf0cb068 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:10.782883 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-194e1be4-a9dc-46f7-b3d0-754bbf0cb068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:10.957220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1469/2914] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:10.965330 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd251d2-806c-46b3-9ead-5e57f8c06c66 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:10.965822 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd251d2-806c-46b3-9ead-5e57f8c06c66 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:10.966403 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd251d2-806c-46b3-9ead-5e57f8c06c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:11.038298 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8fd251d2-806c-46b3-9ead-5e57f8c06c66 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:11.043740 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8fd251d2-806c-46b3-9ead-5e57f8c06c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:11.081154 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1446/2915] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:33:11.083470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e1be4-a9dc-46f7-b3d0-754bbf0cb068 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:11.105637 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e1be4-a9dc-46f7-b3d0-754bbf0cb068 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:11.122691 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df6a9-025b-4618-8bec-1996aa5e9082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:11.126077 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df6a9-025b-4618-8bec-1996aa5e9082 admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:11.126979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df6a9-025b-4618-8bec-1996aa5e9082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:11.127522 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df6a9-025b-4618-8bec-1996aa5e9082 admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:11.132849 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99df6a9-025b-4618-8bec-1996aa5e9082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:33:11.134321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99df6a9-025b-4618-8bec-1996aa5e9082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:11.136064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99df6a9-025b-4618-8bec-1996aa5e9082 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:11.168855 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1470/2916] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:33:11.170881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd251d2-806c-46b3-9ead-5e57f8c06c66 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:11.176648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:11.203521 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd251d2-806c-46b3-9ead-5e57f8c06c66 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:11.237547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43fe83-7f8d-47a7-8ae9-46aa877f2ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:11.238188 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43fe83-7f8d-47a7-8ae9-46aa877f2ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:11.238468 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43fe83-7f8d-47a7-8ae9-46aa877f2ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:11.238738 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43fe83-7f8d-47a7-8ae9-46aa877f2ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:11.239392 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43fe83-7f8d-47a7-8ae9-46aa877f2ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:11.242150 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43fe83-7f8d-47a7-8ae9-46aa877f2ae2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:11.328701 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1447/2917] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:33:11 2022] POST /identity/v3/projects => generated 409 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:11.333231 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df6a9-025b-4618-8bec-1996aa5e9082 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:11.350226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df6a9-025b-4618-8bec-1996aa5e9082 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:11.543558 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9167-452e-47b7-927c-8aa9b54492e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:11.544648 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9167-452e-47b7-927c-8aa9b54492e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:11.545346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9167-452e-47b7-927c-8aa9b54492e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:11.546058 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9167-452e-47b7-927c-8aa9b54492e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:11.547149 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9167-452e-47b7-927c-8aa9b54492e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:11.549491 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9167-452e-47b7-927c-8aa9b54492e0 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:11.598247 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43fe83-7f8d-47a7-8ae9-46aa877f2ae2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:11.598902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1471/2918] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:33:11 2022] POST /identity/v3/users => generated 305 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:11.622698 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43fe83-7f8d-47a7-8ae9-46aa877f2ae2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:11.656071 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9d22d-6968-4aa8-9007-98b83179567c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:11.657197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9d22d-6968-4aa8-9007-98b83179567c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:11.657705 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9d22d-6968-4aa8-9007-98b83179567c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:11.658163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9d22d-6968-4aa8-9007-98b83179567c admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:11.659178 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9d22d-6968-4aa8-9007-98b83179567c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:11.662571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9d22d-6968-4aa8-9007-98b83179567c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:11.688626 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1448/2919] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:11.693005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9167-452e-47b7-927c-8aa9b54492e0 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:11.708741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9167-452e-47b7-927c-8aa9b54492e0 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:11.716786 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35100220-4cac-49c6-b699-640046ec406f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:11.718050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35100220-4cac-49c6-b699-640046ec406f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:11.718550 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35100220-4cac-49c6-b699-640046ec406f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:11.719005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35100220-4cac-49c6-b699-640046ec406f admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:11.719846 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35100220-4cac-49c6-b699-640046ec406f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:11.722453 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35100220-4cac-49c6-b699-640046ec406f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:11.743258 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1472/2920] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:11.745424 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9d22d-6968-4aa8-9007-98b83179567c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:11.756748 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9d22d-6968-4aa8-9007-98b83179567c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:11.763219 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328d87e-1c8e-4fd3-814f-4d5e098352bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:11.763904 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d87e-1c8e-4fd3-814f-4d5e098352bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:11.764197 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d87e-1c8e-4fd3-814f-4d5e098352bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:11.764469 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d87e-1c8e-4fd3-814f-4d5e098352bd admin admin] PATH_INFO: `/v3/projects/9d53be83971d4c0c9eab80ed272242aa/users/a1d9fe79e6424179933ed220db37a3bd/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:11.799989 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328d87e-1c8e-4fd3-814f-4d5e098352bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d53be83971d4c0c9eab80ed272242aa, user_id=a1d9fe79e6424179933ed220db37a3bd, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:11.802790 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328d87e-1c8e-4fd3-814f-4d5e098352bd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:11.844489 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1449/2921] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:11 2022] PUT /identity/v3/projects/9d53be83971d4c0c9eab80ed272242aa/users/a1d9fe79e6424179933ed220db37a3bd/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:11.855606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18439e31-d462-4fa4-aad7-676caae65dd3 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:11.855928 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18439e31-d462-4fa4-aad7-676caae65dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:11.856158 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18439e31-d462-4fa4-aad7-676caae65dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:11.903790 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-18439e31-d462-4fa4-aad7-676caae65dd3 None None] MFA Rules not processed for user `a1d9fe79e6424179933ed220db37a3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:11.905155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-18439e31-d462-4fa4-aad7-676caae65dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:11.910787 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:11.988164 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1473/2922] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:11.992587 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328d87e-1c8e-4fd3-814f-4d5e098352bd admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:12.070852 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328d87e-1c8e-4fd3-814f-4d5e098352bd admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:12.076729 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dff076-8779-49f5-9097-7b4bb7f6ef3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:12.077293 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dff076-8779-49f5-9097-7b4bb7f6ef3c admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:12.077577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dff076-8779-49f5-9097-7b4bb7f6ef3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:12.078103 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dff076-8779-49f5-9097-7b4bb7f6ef3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:12.078746 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dff076-8779-49f5-9097-7b4bb7f6ef3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:12.080840 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dff076-8779-49f5-9097-7b4bb7f6ef3c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:12.127225 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1450/2923] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:33:11 2022] POST /identity/v3/projects => generated 407 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:12.129073 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18439e31-d462-4fa4-aad7-676caae65dd3 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:12.152995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18439e31-d462-4fa4-aad7-676caae65dd3 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:12.292346 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e56965-045e-430f-a4d3-4114cfb95b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:12.293094 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e56965-045e-430f-a4d3-4114cfb95b8f admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:12.293481 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e56965-045e-430f-a4d3-4114cfb95b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:12.293666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e56965-045e-430f-a4d3-4114cfb95b8f admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:12.294297 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e56965-045e-430f-a4d3-4114cfb95b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:12.296287 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e56965-045e-430f-a4d3-4114cfb95b8f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:12.347030 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1474/2924] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:33:12 2022] POST /identity/v3/users => generated 310 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:33:12.350909 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dff076-8779-49f5-9097-7b4bb7f6ef3c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:12.362297 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dff076-8779-49f5-9097-7b4bb7f6ef3c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:12.378011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd31be-a8ef-4680-9330-0f6b13a5ef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:12.378011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd31be-a8ef-4680-9330-0f6b13a5ef3e admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:12.378011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd31be-a8ef-4680-9330-0f6b13a5ef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:12.378011 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd31be-a8ef-4680-9330-0f6b13a5ef3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:12.378425 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd31be-a8ef-4680-9330-0f6b13a5ef3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:12.378915 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd31be-a8ef-4680-9330-0f6b13a5ef3e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:12.404349 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1451/2925] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:12.408925 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e56965-045e-430f-a4d3-4114cfb95b8f admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:12.428517 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e56965-045e-430f-a4d3-4114cfb95b8f admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:12.457296 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feed9f-3c6a-47c4-9ccf-77d022ad0286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:12.458534 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feed9f-3c6a-47c4-9ccf-77d022ad0286 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:12.461690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feed9f-3c6a-47c4-9ccf-77d022ad0286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:12.464775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feed9f-3c6a-47c4-9ccf-77d022ad0286 admin admin] PATH_INFO: `/v3/projects/ba5313dae7b0447c985dfa0afccd5353/users/4b4247adfcbd4aff97a74b4ca1a09552/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:12.497894 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feed9f-3c6a-47c4-9ccf-77d022ad0286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5313dae7b0447c985dfa0afccd5353, user_id=4b4247adfcbd4aff97a74b4ca1a09552, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:12.501460 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feed9f-3c6a-47c4-9ccf-77d022ad0286 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:12.524295 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1475/2926] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:12 2022] PUT /identity/v3/projects/ba5313dae7b0447c985dfa0afccd5353/users/4b4247adfcbd4aff97a74b4ca1a09552/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:12.528456 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd31be-a8ef-4680-9330-0f6b13a5ef3e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:12.564554 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd31be-a8ef-4680-9330-0f6b13a5ef3e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:12.744246 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710714de-8045-4afd-8034-fe46794fd656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:12.744973 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710714de-8045-4afd-8034-fe46794fd656 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:12.745264 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710714de-8045-4afd-8034-fe46794fd656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:12.745539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710714de-8045-4afd-8034-fe46794fd656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:12.746205 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710714de-8045-4afd-8034-fe46794fd656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:12.748108 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710714de-8045-4afd-8034-fe46794fd656 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:12.762616 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1452/2927] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:12.766456 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feed9f-3c6a-47c4-9ccf-77d022ad0286 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:12.779640 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feed9f-3c6a-47c4-9ccf-77d022ad0286 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:12.795939 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b39271-275e-4783-b837-b5db59913b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:12.796861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b39271-275e-4783-b837-b5db59913b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:12.797155 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b39271-275e-4783-b837-b5db59913b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:12.799025 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b39271-275e-4783-b837-b5db59913b2b admin admin] PATH_INFO: `/v3/projects/ba5313dae7b0447c985dfa0afccd5353/users/4b4247adfcbd4aff97a74b4ca1a09552/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:12.805853 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b39271-275e-4783-b837-b5db59913b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5313dae7b0447c985dfa0afccd5353, user_id=4b4247adfcbd4aff97a74b4ca1a09552, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:12.809200 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b39271-275e-4783-b837-b5db59913b2b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:12.837019 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1476/2928] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:12 2022] PUT /identity/v3/projects/ba5313dae7b0447c985dfa0afccd5353/users/4b4247adfcbd4aff97a74b4ca1a09552/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:12.849049 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c98d6-117f-4394-be57-36ceff9b930d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:12.849906 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c98d6-117f-4394-be57-36ceff9b930d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:12.850196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c98d6-117f-4394-be57-36ceff9b930d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:12.894179 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-fe0c98d6-117f-4394-be57-36ceff9b930d None None] MFA Rules not processed for user `4b4247adfcbd4aff97a74b4ca1a09552`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:12.895399 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fe0c98d6-117f-4394-be57-36ceff9b930d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:12.900203 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:12.997447 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1453/2929] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:12 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:13.007564 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b39271-275e-4783-b837-b5db59913b2b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:13.009021 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-f4b39271-275e-4783-b837-b5db59913b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:13.113078 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b39271-275e-4783-b837-b5db59913b2b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:13.125169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32841dc-b7f3-40b7-806a-d8dd83d3f8a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:13.126150 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32841dc-b7f3-40b7-806a-d8dd83d3f8a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:13.126520 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32841dc-b7f3-40b7-806a-d8dd83d3f8a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:13.126936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32841dc-b7f3-40b7-806a-d8dd83d3f8a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:13.143368 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32841dc-b7f3-40b7-806a-d8dd83d3f8a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:13.143707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32841dc-b7f3-40b7-806a-d8dd83d3f8a9 service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:13.153897 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:13.178218 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1477/2930] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:13 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:33:13.409520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c98d6-117f-4394-be57-36ceff9b930d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:13.426488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c98d6-117f-4394-be57-36ceff9b930d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:13.430577 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bb494-8421-4398-ac6f-f988e9d4e9f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:13.431201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bb494-8421-4398-ac6f-f988e9d4e9f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:13.431482 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bb494-8421-4398-ac6f-f988e9d4e9f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:13.431753 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bb494-8421-4398-ac6f-f988e9d4e9f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:13.509070 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bb494-8421-4398-ac6f-f988e9d4e9f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:13.510681 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bb494-8421-4398-ac6f-f988e9d4e9f2 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:13.527517 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1454/2931] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:13 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:16.336304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32841dc-b7f3-40b7-806a-d8dd83d3f8a9 service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:16.338035 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-c32841dc-b7f3-40b7-806a-d8dd83d3f8a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:16.413441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32841dc-b7f3-40b7-806a-d8dd83d3f8a9 service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:16.417177 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88220-2704-4ce4-9fa0-b71e8caf52f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:16.417788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88220-2704-4ce4-9fa0-b71e8caf52f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:16.418068 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88220-2704-4ce4-9fa0-b71e8caf52f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:16.418423 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88220-2704-4ce4-9fa0-b71e8caf52f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:16.420182 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d2b88220-2704-4ce4-9fa0-b71e8caf52f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:16.529669 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b88220-2704-4ce4-9fa0-b71e8caf52f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:16.531197 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b88220-2704-4ce4-9fa0-b71e8caf52f1 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:16.544992 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1478/2932] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:33:23.517434 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=55 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:26.538312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=54 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:30.762802 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bb494-8421-4398-ac6f-f988e9d4e9f2 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:30.773523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bb494-8421-4398-ac6f-f988e9d4e9f2 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:30.777272 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d400d9e-0043-4988-bdc4-9f3ba94b9094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:30.777942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d400d9e-0043-4988-bdc4-9f3ba94b9094 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:30.778286 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d400d9e-0043-4988-bdc4-9f3ba94b9094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:30.778619 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d400d9e-0043-4988-bdc4-9f3ba94b9094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:30.780421 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9d400d9e-0043-4988-bdc4-9f3ba94b9094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:30.791431 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d400d9e-0043-4988-bdc4-9f3ba94b9094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:30.792901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d400d9e-0043-4988-bdc4-9f3ba94b9094 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:30.800956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1455/2933] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:33.267312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88220-2704-4ce4-9fa0-b71e8caf52f1 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:33.329286 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88220-2704-4ce4-9fa0-b71e8caf52f1 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:33.334056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:33.334690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:33.334958 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:33.335226 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] PATH_INFO: `/v3/users/a1d9fe79e6424179933ed220db37a3bd` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:33.336342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d9fe79e6424179933ed220db37a3bd)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:33.338304 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:33.385919 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d9fe79e6424179933ed220db37a3bd'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.386495 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d9fe79e6424179933ed220db37a3bd'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.394170 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d9fe79e6424179933ed220db37a3bd'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.395169 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d9fe79e6424179933ed220db37a3bd'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.403050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d9fe79e6424179933ed220db37a3bd'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.420568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1479/2934] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:33:33 2022] DELETE /identity/v3/users/a1d9fe79e6424179933ed220db37a3bd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:33.425228 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87a4f2-8d7c-4ada-8a6a-9ae81e828b94 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:33.425694 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87a4f2-8d7c-4ada-8a6a-9ae81e828b94 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:33.425811 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87a4f2-8d7c-4ada-8a6a-9ae81e828b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:33.446472 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9b87a4f2-8d7c-4ada-8a6a-9ae81e828b94 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:33.447769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9b87a4f2-8d7c-4ada-8a6a-9ae81e828b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:33.507163 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1456/2935] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:33.516772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:33.518219 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:33.576379 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6477fc0-81e7-4549-95f4-dac5c3ea477e admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:33.579995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789ec4-9a0e-4a45-a849-f5ce2ce15b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:33.580593 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789ec4-9a0e-4a45-a849-f5ce2ce15b75 service neutron] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:33.580906 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789ec4-9a0e-4a45-a849-f5ce2ce15b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:33.581178 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789ec4-9a0e-4a45-a849-f5ce2ce15b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:33.590046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0789ec4-9a0e-4a45-a849-f5ce2ce15b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:33.591465 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0789ec4-9a0e-4a45-a849-f5ce2ce15b75 service neutron] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:33.600733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1480/2936] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 11:33:33.869387 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87a4f2-8d7c-4ada-8a6a-9ae81e828b94 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:33.870834 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9b87a4f2-8d7c-4ada-8a6a-9ae81e828b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:33.883073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87a4f2-8d7c-4ada-8a6a-9ae81e828b94 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:33.887866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:33.888525 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:33.888976 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:33.889274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] PATH_INFO: `/v3/projects/9d53be83971d4c0c9eab80ed272242aa` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:33.890391 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d53be83971d4c0c9eab80ed272242aa)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:33.892423 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:33.944126 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d53be83971d4c0c9eab80ed272242aa', 'request_id': 'req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.949824 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d53be83971d4c0c9eab80ed272242aa', 'request_id': 'req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.956907 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d53be83971d4c0c9eab80ed272242aa', 'request_id': 'req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.968088 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] The token cache is being invalidate because project 9d53be83971d4c0c9eab80ed272242aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:33.968375 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.969092 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:33.970589 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1457/2937] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:33:33 2022] DELETE /identity/v3/projects/9d53be83971d4c0c9eab80ed272242aa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:33.975207 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789ec4-9a0e-4a45-a849-f5ce2ce15b75 service neutron] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:33.984889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789ec4-9a0e-4a45-a849-f5ce2ce15b75 service neutron] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:34.003658 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:34.044427 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:34.045117 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.045396 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.045666 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] PATH_INFO: `/v3/users/4b4247adfcbd4aff97a74b4ca1a09552` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.046793 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b4247adfcbd4aff97a74b4ca1a09552)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:34.048707 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:34.094699 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b4247adfcbd4aff97a74b4ca1a09552'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.095334 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b4247adfcbd4aff97a74b4ca1a09552'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.102930 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b4247adfcbd4aff97a74b4ca1a09552'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.103634 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b4247adfcbd4aff97a74b4ca1a09552'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.110690 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-ff986831-cfed-4f36-933a-0c30af20fa3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b4247adfcbd4aff97a74b4ca1a09552'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.127769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1481/2938] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:33:33 2022] DELETE /identity/v3/users/4b4247adfcbd4aff97a74b4ca1a09552 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:34.398676 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:34.400201 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:34.419654 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5d386-c98f-4fb8-8908-dd5c1f2197ca admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:34.477769 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:34.478439 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.478576 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.478870 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] PATH_INFO: `/v3/projects/ba5313dae7b0447c985dfa0afccd5353` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.479981 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba5313dae7b0447c985dfa0afccd5353)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:34.482024 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:34.493539 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:34.533603 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba5313dae7b0447c985dfa0afccd5353', 'request_id': 'req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.541580 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba5313dae7b0447c985dfa0afccd5353', 'request_id': 'req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.550127 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba5313dae7b0447c985dfa0afccd5353', 'request_id': 'req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.569223 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] The token cache is being invalidate because project ba5313dae7b0447c985dfa0afccd5353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:34.569516 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.570171 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b62fca57-2252-49d7-9c94-d5f9cd89b9d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:34.571867 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1458/2939] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:33:34 2022] DELETE /identity/v3/projects/ba5313dae7b0447c985dfa0afccd5353 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:34.578555 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d706b-8870-458f-b44c-ac55570c95dc None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.578968 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d706b-8870-458f-b44c-ac55570c95dc None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.579124 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d706b-8870-458f-b44c-ac55570c95dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.600066 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-e47d706b-8870-458f-b44c-ac55570c95dc None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:34.601312 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-e47d706b-8870-458f-b44c-ac55570c95dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:34.662016 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1482/2940] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 11:33:34.669785 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7c1e8-254b-4fdb-ad9d-6e07dd523570 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.670074 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7c1e8-254b-4fdb-ad9d-6e07dd523570 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.670352 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7c1e8-254b-4fdb-ad9d-6e07dd523570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.690498 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8bf7c1e8-254b-4fdb-ad9d-6e07dd523570 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:34.691699 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8bf7c1e8-254b-4fdb-ad9d-6e07dd523570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:34.699860 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1459/2941] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:34.703936 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d706b-8870-458f-b44c-ac55570c95dc None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:34.713829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d706b-8870-458f-b44c-ac55570c95dc None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:34.718309 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a35c83-0b50-4871-b35a-e8ab8cf6879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:34.718898 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a35c83-0b50-4871-b35a-e8ab8cf6879b admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.719176 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a35c83-0b50-4871-b35a-e8ab8cf6879b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.719446 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a35c83-0b50-4871-b35a-e8ab8cf6879b admin admin] PATH_INFO: `/v3/domains` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.720081 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a35c83-0b50-4871-b35a-e8ab8cf6879b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 11:33:34.720383 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a35c83-0b50-4871-b35a-e8ab8cf6879b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:34.722005 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a35c83-0b50-4871-b35a-e8ab8cf6879b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:34.733991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1483/2942] 10.222.0.50 () {58 vars in 1153 bytes} [Thu Sep 15 11:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 11:33:34.737892 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7c1e8-254b-4fdb-ad9d-6e07dd523570 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:34.747383 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7c1e8-254b-4fdb-ad9d-6e07dd523570 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:34.751914 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde0c7a-d996-4f3f-bb55-69bb7a7115f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:34.752494 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde0c7a-d996-4f3f-bb55-69bb7a7115f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.752770 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde0c7a-d996-4f3f-bb55-69bb7a7115f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.753073 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde0c7a-d996-4f3f-bb55-69bb7a7115f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.753691 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde0c7a-d996-4f3f-bb55-69bb7a7115f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:34.755690 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde0c7a-d996-4f3f-bb55-69bb7a7115f3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:34.783821 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1460/2943] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:33:34 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:34.789075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a35c83-0b50-4871-b35a-e8ab8cf6879b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:34.799098 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a35c83-0b50-4871-b35a-e8ab8cf6879b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:34.824950 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:34.858264 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928ad82-9dc9-46bd-b05f-79cacb6b6e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:34.858860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928ad82-9dc9-46bd-b05f-79cacb6b6e1c admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.859135 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928ad82-9dc9-46bd-b05f-79cacb6b6e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.859402 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928ad82-9dc9-46bd-b05f-79cacb6b6e1c admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.860029 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928ad82-9dc9-46bd-b05f-79cacb6b6e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:34.861972 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928ad82-9dc9-46bd-b05f-79cacb6b6e1c admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:34.882721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1484/2944] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:33:34 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 11:33:34.886851 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde0c7a-d996-4f3f-bb55-69bb7a7115f3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:34.897670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde0c7a-d996-4f3f-bb55-69bb7a7115f3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:34.902260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563afba5-8868-41f5-902f-d72349378f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:34.902854 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563afba5-8868-41f5-902f-d72349378f21 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.903134 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563afba5-8868-41f5-902f-d72349378f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.903402 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563afba5-8868-41f5-902f-d72349378f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.904032 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563afba5-8868-41f5-902f-d72349378f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:34.905675 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563afba5-8868-41f5-902f-d72349378f21 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:34.915984 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1461/2945] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:34.919861 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928ad82-9dc9-46bd-b05f-79cacb6b6e1c admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:34.930526 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928ad82-9dc9-46bd-b05f-79cacb6b6e1c admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:34.935389 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43c4b7-01dd-4d04-bbe3-c4c77037b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:34.936113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43c4b7-01dd-4d04-bbe3-c4c77037b11f admin admin] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.936408 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43c4b7-01dd-4d04-bbe3-c4c77037b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.936685 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43c4b7-01dd-4d04-bbe3-c4c77037b11f admin admin] PATH_INFO: `/v3/roles` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.937370 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43c4b7-01dd-4d04-bbe3-c4c77037b11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:34.938991 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43c4b7-01dd-4d04-bbe3-c4c77037b11f admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:34.957307 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1485/2946] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:34.958493 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563afba5-8868-41f5-902f-d72349378f21 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:34.969102 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563afba5-8868-41f5-902f-d72349378f21 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:34.974815 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396922fd-4cae-4a3a-8b31-a64396de40d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:34.975488 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396922fd-4cae-4a3a-8b31-a64396de40d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:34.975777 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396922fd-4cae-4a3a-8b31-a64396de40d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:34.976051 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396922fd-4cae-4a3a-8b31-a64396de40d7 admin admin] PATH_INFO: `/v3/projects/f77b92336ad1431aa8ab06c281ff526e/users/53e608442c36425482907f674fd93ef2/roles/4cef964ca8d143eaadbb479db64250f1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:34.992233 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396922fd-4cae-4a3a-8b31-a64396de40d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77b92336ad1431aa8ab06c281ff526e, user_id=53e608442c36425482907f674fd93ef2, role_id=4cef964ca8d143eaadbb479db64250f1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:34.995040 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396922fd-4cae-4a3a-8b31-a64396de40d7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:35.020614 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1462/2947] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:34 2022] PUT /identity/v3/projects/f77b92336ad1431aa8ab06c281ff526e/users/53e608442c36425482907f674fd93ef2/roles/4cef964ca8d143eaadbb479db64250f1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:35.032129 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebfc28-87bd-4843-9d71-43bfebb54bb8 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.032421 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebfc28-87bd-4843-9d71-43bfebb54bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.032721 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebfc28-87bd-4843-9d71-43bfebb54bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.062464 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-abebfc28-87bd-4843-9d71-43bfebb54bb8 None None] MFA Rules not processed for user `53e608442c36425482907f674fd93ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:35.063692 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-abebfc28-87bd-4843-9d71-43bfebb54bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:35.131141 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1486/2948] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:35.135886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396922fd-4cae-4a3a-8b31-a64396de40d7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:35.201571 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396922fd-4cae-4a3a-8b31-a64396de40d7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:35.206735 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de8ea-5051-4691-ae73-1f9fe2c4e47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:35.207340 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de8ea-5051-4691-ae73-1f9fe2c4e47f admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.208847 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de8ea-5051-4691-ae73-1f9fe2c4e47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.209132 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de8ea-5051-4691-ae73-1f9fe2c4e47f admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.210141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de8ea-5051-4691-ae73-1f9fe2c4e47f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:35.212365 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de8ea-5051-4691-ae73-1f9fe2c4e47f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:35.241520 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1463/2949] 10.222.0.50 () {60 vars in 1153 bytes} [Thu Sep 15 11:33:35 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:35.246342 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebfc28-87bd-4843-9d71-43bfebb54bb8 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:35.257486 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebfc28-87bd-4843-9d71-43bfebb54bb8 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:35.314220 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb45d2d-a11f-4f14-8876-4ee4d93a4eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:35.314825 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb45d2d-a11f-4f14-8876-4ee4d93a4eda admin admin] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.315109 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb45d2d-a11f-4f14-8876-4ee4d93a4eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.315385 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb45d2d-a11f-4f14-8876-4ee4d93a4eda admin admin] PATH_INFO: `/v3/users` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.316018 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb45d2d-a11f-4f14-8876-4ee4d93a4eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:35.318050 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb45d2d-a11f-4f14-8876-4ee4d93a4eda admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:35.339737 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1487/2950] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Sep 15 11:33:35 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 11:33:35.343751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de8ea-5051-4691-ae73-1f9fe2c4e47f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:35.354943 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de8ea-5051-4691-ae73-1f9fe2c4e47f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:35.359788 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e9e63-3c1b-40e2-8fea-6765e8e2e85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:35.360374 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e9e63-3c1b-40e2-8fea-6765e8e2e85e admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.360653 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e9e63-3c1b-40e2-8fea-6765e8e2e85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.360987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e9e63-3c1b-40e2-8fea-6765e8e2e85e admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.361623 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e9e63-3c1b-40e2-8fea-6765e8e2e85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:35.363260 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e9e63-3c1b-40e2-8fea-6765e8e2e85e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:35.374452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1464/2951] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:35.378586 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb45d2d-a11f-4f14-8876-4ee4d93a4eda admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:35.388715 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb45d2d-a11f-4f14-8876-4ee4d93a4eda admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:35.393270 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd69d55-69c6-485d-af8d-636facd2b127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:35.393910 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd69d55-69c6-485d-af8d-636facd2b127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.394186 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd69d55-69c6-485d-af8d-636facd2b127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.394455 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd69d55-69c6-485d-af8d-636facd2b127 admin admin] PATH_INFO: `/v3/projects/59dd149b5b614c1ca97612a18337cfcd/users/a13dc81107164162991ccafdc51092a7/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.409060 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd69d55-69c6-485d-af8d-636facd2b127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59dd149b5b614c1ca97612a18337cfcd, user_id=a13dc81107164162991ccafdc51092a7, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:35.412620 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd69d55-69c6-485d-af8d-636facd2b127 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:35.425303 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1488/2952] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:35 2022] PUT /identity/v3/projects/59dd149b5b614c1ca97612a18337cfcd/users/a13dc81107164162991ccafdc51092a7/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 11:33:35.429774 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e9e63-3c1b-40e2-8fea-6765e8e2e85e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:35.439966 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e9e63-3c1b-40e2-8fea-6765e8e2e85e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:35.447862 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:35.499808 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b44748-eb54-4320-af61-db9cdcbd0d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:35.500435 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44748-eb54-4320-af61-db9cdcbd0d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.500581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44748-eb54-4320-af61-db9cdcbd0d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.500877 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44748-eb54-4320-af61-db9cdcbd0d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.501501 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b44748-eb54-4320-af61-db9cdcbd0d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:35.503094 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b44748-eb54-4320-af61-db9cdcbd0d2f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:35.512673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1465/2953] 10.222.0.50 () {58 vars in 1122 bytes} [Thu Sep 15 11:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 11:33:35.516979 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd69d55-69c6-485d-af8d-636facd2b127 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:35.522896 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:35.529367 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd69d55-69c6-485d-af8d-636facd2b127 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:35.534319 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe6a2c-5ba9-4b0f-bad8-f721787e42b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:35.535156 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe6a2c-5ba9-4b0f-bad8-f721787e42b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.535445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe6a2c-5ba9-4b0f-bad8-f721787e42b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.536318 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe6a2c-5ba9-4b0f-bad8-f721787e42b7 admin admin] PATH_INFO: `/v3/projects/59dd149b5b614c1ca97612a18337cfcd/users/a13dc81107164162991ccafdc51092a7/roles/0be9777002c641b4a47f78df647cc173` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.538373 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe6a2c-5ba9-4b0f-bad8-f721787e42b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59dd149b5b614c1ca97612a18337cfcd, user_id=a13dc81107164162991ccafdc51092a7, role_id=0be9777002c641b4a47f78df647cc173)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:35.541071 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe6a2c-5ba9-4b0f-bad8-f721787e42b7 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:35.551608 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1489/2954] 10.222.0.50 () {60 vars in 1483 bytes} [Thu Sep 15 11:33:35 2022] PUT /identity/v3/projects/59dd149b5b614c1ca97612a18337cfcd/users/a13dc81107164162991ccafdc51092a7/roles/0be9777002c641b4a47f78df647cc173 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:35.562452 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cfe52-34da-476b-9858-c2f9085b16b2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.562764 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cfe52-34da-476b-9858-c2f9085b16b2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.563043 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cfe52-34da-476b-9858-c2f9085b16b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.590942 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-101cfe52-34da-476b-9858-c2f9085b16b2 None None] MFA Rules not processed for user `a13dc81107164162991ccafdc51092a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:35.592190 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-101cfe52-34da-476b-9858-c2f9085b16b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:35.665045 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1466/2955] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:35.678705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe6a2c-5ba9-4b0f-bad8-f721787e42b7 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:35.680775 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-fcfe6a2c-5ba9-4b0f-bad8-f721787e42b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:35.750597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe6a2c-5ba9-4b0f-bad8-f721787e42b7 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:35.754213 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96048c16-8ca0-4b25-958f-c0ee0400100f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:35.754994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048c16-8ca0-4b25-958f-c0ee0400100f service cinder] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.755362 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048c16-8ca0-4b25-958f-c0ee0400100f service cinder] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.755670 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048c16-8ca0-4b25-958f-c0ee0400100f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.766426 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96048c16-8ca0-4b25-958f-c0ee0400100f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:35.767965 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96048c16-8ca0-4b25-958f-c0ee0400100f service cinder] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:35.779372 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1490/2956] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:35.861759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cfe52-34da-476b-9858-c2f9085b16b2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:35.871463 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cfe52-34da-476b-9858-c2f9085b16b2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:35.875065 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da342b1f-261c-4ff9-a658-3b0f14b724d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7fa5d550a2d442a820cf4e12ebc36c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:35.875665 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da342b1f-261c-4ff9-a658-3b0f14b724d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:35.875944 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da342b1f-261c-4ff9-a658-3b0f14b724d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:35.876215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da342b1f-261c-4ff9-a658-3b0f14b724d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:35.932658 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da342b1f-261c-4ff9-a658-3b0f14b724d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:35.934141 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da342b1f-261c-4ff9-a658-3b0f14b724d2 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:35.943703 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1467/2957] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:37.118871 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96048c16-8ca0-4b25-958f-c0ee0400100f service cinder] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:37.120577 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-96048c16-8ca0-4b25-958f-c0ee0400100f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:37.200056 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96048c16-8ca0-4b25-958f-c0ee0400100f service cinder] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:37.205590 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3b879-3543-4c13-96c8-61c2e287dc00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:37.206206 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3b879-3543-4c13-96c8-61c2e287dc00 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:37.206489 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3b879-3543-4c13-96c8-61c2e287dc00 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:37.206767 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3b879-3543-4c13-96c8-61c2e287dc00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:37.208571 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b5f3b879-3543-4c13-96c8-61c2e287dc00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:37.220568 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3b879-3543-4c13-96c8-61c2e287dc00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:37.222194 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3b879-3543-4c13-96c8-61c2e287dc00 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:37.232944 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1491/2958] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:41.894027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da342b1f-261c-4ff9-a658-3b0f14b724d2 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:41.943562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:41.975562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da342b1f-261c-4ff9-a658-3b0f14b724d2 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:41.980310 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:41.980985 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:41.981265 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:41.981534 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] PATH_INFO: `/v3/users/53e608442c36425482907f674fd93ef2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:41.982657 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e608442c36425482907f674fd93ef2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:41.985874 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:42.033384 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e608442c36425482907f674fd93ef2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.041105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e608442c36425482907f674fd93ef2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.041726 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e608442c36425482907f674fd93ef2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.048886 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e608442c36425482907f674fd93ef2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.049543 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e608442c36425482907f674fd93ef2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.067763 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1468/2959] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:33:41 2022] DELETE /identity/v3/users/53e608442c36425482907f674fd93ef2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:42.072232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93428148-c82a-4569-ae67-17fdd9c36bc2 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:42.072748 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93428148-c82a-4569-ae67-17fdd9c36bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:42.072864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93428148-c82a-4569-ae67-17fdd9c36bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:42.092895 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-93428148-c82a-4569-ae67-17fdd9c36bc2 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:42.094101 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-93428148-c82a-4569-ae67-17fdd9c36bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:42.152860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1492/2960] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:42.163267 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:42.164670 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:42.221866 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b423db3-3329-4935-8bbc-07b3c76f6835 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:42.225591 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c603cd-b958-4bb0-a060-55283d067625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:42.226196 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c603cd-b958-4bb0-a060-55283d067625 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:42.226476 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c603cd-b958-4bb0-a060-55283d067625 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:42.226751 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c603cd-b958-4bb0-a060-55283d067625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:42.235673 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c603cd-b958-4bb0-a060-55283d067625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:42.237142 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c603cd-b958-4bb0-a060-55283d067625 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:42.247185 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1469/2961] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:42.512388 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93428148-c82a-4569-ae67-17fdd9c36bc2 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:42.513860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-93428148-c82a-4569-ae67-17fdd9c36bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:42.525902 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93428148-c82a-4569-ae67-17fdd9c36bc2 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:42.530524 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:42.531166 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:42.531445 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:42.531716 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] PATH_INFO: `/v3/projects/f77b92336ad1431aa8ab06c281ff526e` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:42.532891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77b92336ad1431aa8ab06c281ff526e)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:42.534816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:42.547435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:42.586037 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77b92336ad1431aa8ab06c281ff526e', 'request_id': 'req-6186dd6a-4f4b-449e-ae85-d4a075cc9367'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.591551 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77b92336ad1431aa8ab06c281ff526e', 'request_id': 'req-6186dd6a-4f4b-449e-ae85-d4a075cc9367'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.598606 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77b92336ad1431aa8ab06c281ff526e', 'request_id': 'req-6186dd6a-4f4b-449e-ae85-d4a075cc9367'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.609535 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] The token cache is being invalidate because project f77b92336ad1431aa8ab06c281ff526e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:42.609817 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.610435 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.611900 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1493/2962] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:33:42 2022] DELETE /identity/v3/projects/f77b92336ad1431aa8ab06c281ff526e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:42.616337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c603cd-b958-4bb0-a060-55283d067625 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:42.625931 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c603cd-b958-4bb0-a060-55283d067625 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:42.677025 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:42.688923 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:42.689761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:42.690044 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:42.690315 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] PATH_INFO: `/v3/users/a13dc81107164162991ccafdc51092a7` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:42.691562 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13dc81107164162991ccafdc51092a7)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:42.693881 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:42.744480 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13dc81107164162991ccafdc51092a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.751956 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13dc81107164162991ccafdc51092a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.752630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13dc81107164162991ccafdc51092a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.759987 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13dc81107164162991ccafdc51092a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.760645 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13dc81107164162991ccafdc51092a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:42.778028 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1470/2963] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:33:42 2022] DELETE /identity/v3/users/a13dc81107164162991ccafdc51092a7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:43.071214 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:43.073064 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:43.093719 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186dd6a-4f4b-449e-ae85-d4a075cc9367 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:43.161829 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:43.162462 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:43.162756 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:43.162994 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] PATH_INFO: `/v3/projects/59dd149b5b614c1ca97612a18337cfcd` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:43.164113 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=59dd149b5b614c1ca97612a18337cfcd)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:43.166047 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:43.213940 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59dd149b5b614c1ca97612a18337cfcd', 'request_id': 'req-d115b0b2-3851-4814-a162-a13a6035bd0b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:43.219723 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59dd149b5b614c1ca97612a18337cfcd', 'request_id': 'req-d115b0b2-3851-4814-a162-a13a6035bd0b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:43.227953 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59dd149b5b614c1ca97612a18337cfcd', 'request_id': 'req-d115b0b2-3851-4814-a162-a13a6035bd0b'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:43.239952 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] The token cache is being invalidate because project 59dd149b5b614c1ca97612a18337cfcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:43.240232 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:43.240889 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:43.242365 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1494/2964] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:33:43 2022] DELETE /identity/v3/projects/59dd149b5b614c1ca97612a18337cfcd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:49.643993 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:49.645664 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:49.702522 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166aeddd-f774-4ed2-9fca-2d721377a2a4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:49.751318 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:49.751924 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:49.752257 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:49.752528 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:49.754549 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:49.799899 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:49.823901 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:49.825410 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:49.836098 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1471/2965] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:51.434139 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:51.436133 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:51.479505 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:51.533411 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b0b2-3851-4814-a162-a13a6035bd0b admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:51.539860 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40275f7-f2f9-4c2b-9227-a8ae5afa8fb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '218a015dd2b34435a76ecc8a763562ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:51.540788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40275f7-f2f9-4c2b-9227-a8ae5afa8fb5 service nova] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:51.540788 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40275f7-f2f9-4c2b-9227-a8ae5afa8fb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:51.541031 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40275f7-f2f9-4c2b-9227-a8ae5afa8fb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:51.542761 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-d40275f7-f2f9-4c2b-9227-a8ae5afa8fb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:51.567717 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40275f7-f2f9-4c2b-9227-a8ae5afa8fb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:51.570835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40275f7-f2f9-4c2b-9227-a8ae5afa8fb5 service nova] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:51.585984 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1495/2966] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:52.999825 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:53.001413 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:53.096019 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321cec9-4612-4ee6-8969-bac20b75a3f4 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:53.105236 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:53.106050 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:53.106337 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:53.106611 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] PATH_INFO: `/v3/users/3bfc300aefab4a22a949d4807073d5d1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:53.107783 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bfc300aefab4a22a949d4807073d5d1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:53.109830 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:53.161974 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bfc300aefab4a22a949d4807073d5d1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.171552 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bfc300aefab4a22a949d4807073d5d1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.172283 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bfc300aefab4a22a949d4807073d5d1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.180829 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bfc300aefab4a22a949d4807073d5d1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.181523 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bfc300aefab4a22a949d4807073d5d1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.201750 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1472/2967] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:33:52 2022] DELETE /identity/v3/users/3bfc300aefab4a22a949d4807073d5d1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:33:53.205976 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9fdb7-6cb8-4144-8cba-67dce7bfb129 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:53.206441 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9fdb7-6cb8-4144-8cba-67dce7bfb129 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:53.206872 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9fdb7-6cb8-4144-8cba-67dce7bfb129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:53.228381 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-31f9fdb7-6cb8-4144-8cba-67dce7bfb129 None None] MFA Rules not processed for user `c62c262317454775845cb45d452c8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:33:53.229835 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-31f9fdb7-6cb8-4144-8cba-67dce7bfb129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:53.291741 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1496/2968] 10.222.0.50 () {58 vars in 958 bytes} [Thu Sep 15 11:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:33:53.302547 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:53.304115 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:53.362781 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24e9e2-9244-4c54-9b5f-7bc958cbacc6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:53.367026 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0c1c5-7e73-4831-af96-b3000ed423c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5e7b08dc3544e76ac6732ca2aa89bf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:53.367636 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0c1c5-7e73-4831-af96-b3000ed423c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:53.367918 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0c1c5-7e73-4831-af96-b3000ed423c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:53.368198 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0c1c5-7e73-4831-af96-b3000ed423c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:53.377648 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0c1c5-7e73-4831-af96-b3000ed423c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:53.379183 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0c1c5-7e73-4831-af96-b3000ed423c3 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:53.384252 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:33:53.389667 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1473/2969] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:33:53.677891 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9fdb7-6cb8-4144-8cba-67dce7bfb129 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:33:53.679376 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-31f9fdb7-6cb8-4144-8cba-67dce7bfb129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:33:53.692333 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9fdb7-6cb8-4144-8cba-67dce7bfb129 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:33:53.697819 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:33:53.698466 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:33:53.698744 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:33:53.699012 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] PATH_INFO: `/v3/projects/3aebb788805e42deb30b647dd53e4c62` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:33:53.700168 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aebb788805e42deb30b647dd53e4c62)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:33:53.702614 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:33:53.759705 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aebb788805e42deb30b647dd53e4c62', 'request_id': 'req-4f584990-9767-4e56-9539-7ec4671c4a9d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.765864 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aebb788805e42deb30b647dd53e4c62', 'request_id': 'req-4f584990-9767-4e56-9539-7ec4671c4a9d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.774733 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aebb788805e42deb30b647dd53e4c62', 'request_id': 'req-4f584990-9767-4e56-9539-7ec4671c4a9d'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.786796 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] The token cache is being invalidate because project 3aebb788805e42deb30b647dd53e4c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:33:53.787089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.787770 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-4f584990-9767-4e56-9539-7ec4671c4a9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:33:53.789301 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1497/2970] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:33:53 2022] DELETE /identity/v3/projects/3aebb788805e42deb30b647dd53e4c62 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:34:03.783075 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:35:50.930939 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0c1c5-7e73-4831-af96-b3000ed423c3 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:35:50.932338 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0db0c1c5-7e73-4831-af96-b3000ed423c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:35:51.034714 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0c1c5-7e73-4831-af96-b3000ed423c3 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:35:51.094215 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:35:51.094813 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:35:51.095091 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:35:51.095359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:35:51.097083 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:35:51.107922 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:35:51.109581 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:35:51.119394 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1474/2971] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:35:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:36:01.115162 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=71 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:36:02.527401 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917eacc3-0e04-451e-a807-84dfcbbf3d32 None None] REQUEST_METHOD: `POST` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:36:02.527847 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917eacc3-0e04-451e-a807-84dfcbbf3d32 None None] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:36:02.527949 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917eacc3-0e04-451e-a807-84dfcbbf3d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:36:02.548470 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.auth.core [None req-917eacc3-0e04-451e-a807-84dfcbbf3d32 None None] MFA Rules not processed for user `0991767dd90d4f62bfe5484aad435669`. Rule list: `[]` (Enabled: `True`). {{(pid=81388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 11:36:02.549714 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-917eacc3-0e04-451e-a807-84dfcbbf3d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:36:02.606089 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1498/2972] 10.222.0.50 () {58 vars in 967 bytes} [Thu Sep 15 11:36:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 11:36:02.647004 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:36:02.648409 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:36:02.716263 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c185a-f17d-4519-97c0-082f49e2c7d2 service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:36:02.721405 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e7edc-4d41-4727-a105-eefda420cddb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06007b3b2260409f863ac3d0ac85f4cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:36:02.722003 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e7edc-4d41-4727-a105-eefda420cddb service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:36:02.722284 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e7edc-4d41-4727-a105-eefda420cddb service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:36:02.722556 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e7edc-4d41-4727-a105-eefda420cddb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:36:02.730761 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e7edc-4d41-4727-a105-eefda420cddb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:36:02.732359 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e7edc-4d41-4727-a105-eefda420cddb service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:36:02.741105 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1475/2973] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Sep 15 11:36:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 11:36:09.112615 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917eacc3-0e04-451e-a807-84dfcbbf3d32 None None] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:36:09.114281 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-917eacc3-0e04-451e-a807-84dfcbbf3d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:36:09.183957 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917eacc3-0e04-451e-a807-84dfcbbf3d32 None None] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:36:09.188574 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:36:09.189259 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:36:09.189536 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:36:09.189803 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] PATH_INFO: `/v3/users/53d8bbb16c794d13bf005e21b832c87e` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:36:09.190922 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d8bbb16c794d13bf005e21b832c87e)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:36:09.193618 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:36:09.244771 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d8bbb16c794d13bf005e21b832c87e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.245447 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d8bbb16c794d13bf005e21b832c87e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.253510 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d8bbb16c794d13bf005e21b832c87e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.254138 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d8bbb16c794d13bf005e21b832c87e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.261110 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d8bbb16c794d13bf005e21b832c87e'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.278391 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1499/2974] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:36:09 2022] DELETE /identity/v3/users/53d8bbb16c794d13bf005e21b832c87e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:36:09.452737 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e7edc-4d41-4727-a105-eefda420cddb service swift] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:36:09.463417 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e7edc-4d41-4727-a105-eefda420cddb service swift] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:36:09.521741 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:36:09.522372 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:36:09.522649 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:36:09.522921 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] PATH_INFO: `/v3/projects/5bebc3459c1c4e5f8e766d90f0859b16` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:36:09.524027 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bebc3459c1c4e5f8e766d90f0859b16)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:36:09.525977 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:36:09.575113 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bebc3459c1c4e5f8e766d90f0859b16', 'request_id': 'req-a65e6256-bd45-4434-aa0c-5f05399bb888'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.580550 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bebc3459c1c4e5f8e766d90f0859b16', 'request_id': 'req-a65e6256-bd45-4434-aa0c-5f05399bb888'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.587856 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bebc3459c1c4e5f8e766d90f0859b16', 'request_id': 'req-a65e6256-bd45-4434-aa0c-5f05399bb888'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.599940 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] The token cache is being invalidate because project 5bebc3459c1c4e5f8e766d90f0859b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:36:09.600224 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.600875 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.602608 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1476/2975] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:36:09 2022] DELETE /identity/v3/projects/5bebc3459c1c4e5f8e766d90f0859b16 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:36:09.606329 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:36:09.607713 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:36:09.621487 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c6b85-f62c-467e-a31e-e6edf4f43a02 admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:36:09.640877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:36:09.681862 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:36:09.682492 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:36:09.682772 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:36:09.683046 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] PATH_INFO: `/v3/users/7f7ec5c791134db082cbb362e56e92da` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:36:09.684504 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7ec5c791134db082cbb362e56e92da)` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:36:09.686907 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:36:09.732321 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7ec5c791134db082cbb362e56e92da'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.732995 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7ec5c791134db082cbb362e56e92da'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.740816 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7ec5c791134db082cbb362e56e92da'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.741432 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7ec5c791134db082cbb362e56e92da'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.748327 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.notifications [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7ec5c791134db082cbb362e56e92da'} {{(pid=81388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:09.765279 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1500/2976] 10.222.0.50 () {58 vars in 1224 bytes} [Thu Sep 15 11:36:09 2022] DELETE /identity/v3/users/7f7ec5c791134db082cbb362e56e92da => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:36:09.937826 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:36:09.939274 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:36:09.952164 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e6256-bd45-4434-aa0c-5f05399bb888 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:36:09.972756 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:36:10.015429 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c62c262317454775845cb45d452c8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f07107ef09454996be197ead3ddf41', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:36:10.016629 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:36:10.016945 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:36:10.017222 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] PATH_INFO: `/v3/projects/23336ce1c1e74b7693d45ac090b2b935` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:36:10.018366 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23336ce1c1e74b7693d45ac090b2b935)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:36:10.020291 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:36:10.067012 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23336ce1c1e74b7693d45ac090b2b935', 'request_id': 'req-5450cfbe-c528-406e-8d14-9e3410d93b52'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:10.072586 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23336ce1c1e74b7693d45ac090b2b935', 'request_id': 'req-5450cfbe-c528-406e-8d14-9e3410d93b52'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:10.080707 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23336ce1c1e74b7693d45ac090b2b935', 'request_id': 'req-5450cfbe-c528-406e-8d14-9e3410d93b52'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:10.091484 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] The token cache is being invalidate because project 23336ce1c1e74b7693d45ac090b2b935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 11:36:10.091759 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:10.092466 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5450cfbe-c528-406e-8d14-9e3410d93b52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 11:36:10.094630 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1477/2977] 10.222.0.50 () {58 vars in 1233 bytes} [Thu Sep 15 11:36:09 2022] DELETE /identity/v3/projects/23336ce1c1e74b7693d45ac090b2b935 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 11:36:19.761890 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:36:20.088508 dv-cinder-824149-8-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 11:40:53.886417 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] Authenticating user token {{(pid=81388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 11:40:53.887863 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:40:53.997983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c909366-1845-4d8c-a4cf-d69da54400ee admin admin] Validating token access rules against request {{(pid=81388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 11:40:54.056983 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812c5ae-03de-4bdc-8789-639f83866b96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0620b2af88034caa8de99e9b48b05f7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f1eafaf73ac46c68507d601a32598f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 11:40:54.057597 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812c5ae-03de-4bdc-8789-639f83866b96 service placement] REQUEST_METHOD: `GET` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 11:40:54.057877 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812c5ae-03de-4bdc-8789-639f83866b96 service placement] SCRIPT_NAME: `/identity` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 11:40:54.058144 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812c5ae-03de-4bdc-8789-639f83866b96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 11:40:54.059840 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.fernet_utils [None req-b812c5ae-03de-4bdc-8789-639f83866b96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 11:40:54.071575 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812c5ae-03de-4bdc-8789-639f83866b96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 11:40:54.073181 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812c5ae-03de-4bdc-8789-639f83866b96 service placement] RBAC: Authorization granted {{(pid=81388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 11:40:54.083769 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: [pid: 81388|app: 0|req: 1501/2978] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Sep 15 11:40:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 11:41:04.079437 dv-cinder-824149-8-iscsi devstack@keystone.service[81388]: DEBUG dbcounter [-] [81388] Writing DB stats keystone:SELECT=76 {{(pid=81388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}